Get the top HN stories in your inbox every day.
diego_moita
breck
I agree and think strong emphasis should be on how a knowledge base saves its files, which should be very interoperable.
I build my own little DSLs and have my own experimental knowledge base studio going, where I can edit things using my editors (Sublime, VSCode and Vim), view/query/visualize in my web browser, and also view/edit in a spreadsheet web interface using Handsontable.
The secret trick is all the DSLs use a thing I work on called Tree Notation (some use an even dumber version simply called Grid Notation). It's a plain text notation that just defines lists of words (aka cells) on a line (aka a node or row) and if you indent a line it becomes a child of the parent line (like python) providing support for tree structures and scope. That's the basics, and then on top of that you can design all sorts of DSLs with types and very different parsing strategies than traditional languages.
For example, here's a Tree Language called Dumdown that compiles to markdown or html: https://jtree.treenotation.org/designer/#standard%20dumbdown
But that's still a relatively traditional language. I have some languages which generally you want to use a spreadsheet interface to write programs in, where you can just plop a tree anywhere on the sheet, and then start writing trees. Those ones are kind of like having a canvas for your project instead of a folder with files. Makes for a good base for building DSLs for simulations.
Sorry, I digress, but my purpose in bringing this up is that if anyone is working on personal knowledge base software like the OP, Tree Notation is a pretty useful thing, if you can figure it out. I've got it figured out but not very good at explaining it.
ar-nelson
Tree Notation looks very interesting. It also looks like just another version of S-expressions (but with indentation instead of parens). That's not a bad thing, though! S-expressions aren't used nearly as much as they should be, and indentation syntax can be much clearer than parens.
What's most interesting is the language design DSL that makes it easy to put together a new tree language. I wonder if that can be implemented in Lisp, possibly with an indentation-sensitive syntax like Wisp[1] so that it retains most of Tree Notation's features.
breck
Yes! It's mostly S-Expressions!
However, the magic happens when you remove the parens and go to 2 and 3 dimensions. Think of Javascript, which has 20 million users, versus Excel, which has 2 billion^1.
To paraphrase Twain, removing one thing can be the difference between a lightning bug and lightning.
[1] Numbers are approximate.
Blahah
This is fascinating. Don't suppose you'd be willing to share your dsl definitions, or can recommend any collections? Seems like a great layer to have in a modular knowledge system.
breck
A few of them are out there in the wild.
I definitely hope to share more in the future but at the moment busy focusing with the team on a huge refactor of our codebase at the day job (which includes a fun new DSL editable in a spreadsheet).
When we ship that and I take a breather hope to give the Tree Notation stuff a refresh and publish some of the newer langs.
warpech
Just saw https://www.youtube.com/watch?v=vn2aJA5ANUc. It looks great! I am excited that Handsontable works for you in this context. Please let me know if you'd need any help with it.
RickS
This is awesome. I just finished with a more narrowly scoped and probably worse version of something very similar – in JS/TS no less.
Looking forward to digging deeper into this later.
zensavona
It probably is a bad idea in a way, but in my own personal experience I have found the biggest hurdle to a knowledge management system being actually used and up to date is _actually using it_, and having it just there in front of me in my IDE would probably go a long way to help with that.
Groxx
I recently switched to Obsidian for my desktop pile-of-notes - so far it's pleasantly compliant with everything else I've done so far, and hasn't done anything to screw up my notes or make it hard to understand in other systems (e.g. Notable has a... "unique" folder system). Definitely recommend giving it a shot.
Blahah
Check out Joplin, Dendron, Notable, BoostNote, Roam Research.
For wiki style check out Agora: https://anagora.org/node/agora
Honourable mention for Notational Velocity which pairs well as a lightweight quick interface with any other md filesystem based note software.
murrayb
+1 for Joplin, I moved over from Evernote because I was worried about Evernotes future and I have been pretty happy with Joplin. It's not quite as featured as Evernote but it is more stable and reliable.
bobajeff
Never heard of Dendron before. Just looked at their site and it's basically the same exact tool as Foam here. I wonder if they're related?
crucialfelix
Dendron is more opinionated. It seems cool, but I didn't want to get too involved in a system.
Foam is just markdown with some vscode extensions. Foam is closer to obsidian.
Blahah
It's similar and they are interoperable, but Dendron runs inside VS Code.
calebkaiser
I felt/feel the same way, which kept me from using Foam for a very long time.
That being said, I've now used nearly everything (TiddlyWiki, Obsidian, even Emacs with Org-Roam) and have recently transitioned to Foam. Foam, for me, is the best option I've used.
A couple of reasons:
- Setup and maintenance are effectively nil, and customizing it is extremely easy. This was the big drawback of org-roam for me. It seems extremely powerful, but even after initial setup, it was just a headache to keep it maintained (and this is not me simply hating products that require config. I'm not particulary sensitive to upkeep--I'm typing this on a Surface Pro 3 running Linux, I'm used to needing to fix things.)
- Graph view/general UI. A big feature for me is the graph view, similar to Roam, and the associated UI (autogenerated backlinks, etc.). Foam is fantastic here. The general note taking experience is much better than TiddlyWiki (which I think is a fantastic tool as well) and I have to think less about constructing a knowledge graph, it just happens. Also, due to the VS Code ecosystem, there are a ton of nice extensions available for diagramming, embedding images, writing in LaTeX, etc. that make note taking nice without introducing overhead.
- Privacy/open source. This is perhaps counter-intuitive, but Foam has been one of the better options on this front. It's all markdown/text/html, as you say, but it's also fully open source, unlike something like Obsidian (which has a fantastic out-of-the-box writing experience, btw). I run Foam on VSCodium, a community-driven project that rebuilds VS Code into freely-licensed binaries with all of MS's telemetry turned off. The Foam maintainers have been open and encouraging about making sure Foam works in VSCodium ecosystem: https://github.com/foambubble/foam/issues/26
My whole setup now is more or less run out of Foam. I push up notes to a Git repo, which triggers a deploy of a little web app I have that renders a site built out of my notes, allowing me to do things like view my todo's or access information from my phone/other devices. It's been really easy to set up and maintain, and the Foam community has consistently impressed me.
madsbuch
One of the absolute most critical features of my own knowledge base is that it is entirely in my control. Hence it would not be possible to use an online platform.
- As Derek Sivers say: Apps come and go, we want to use our knowledge base til we die - I need to know nobody is looking over my shoulders. A bit like social cooling. I will not be honest if I might be sharing.
You mention Obsidian which is definitely a good candidate. But hey, I just use a folder of markdown files. I can use the editor best suited for the task I am doing. So I both use Obsidian, VS Code, normal text editor and whatever floats my boat.
foota
I think they're using vscode here as an application platform of sorts.
smnrchrds
Like Emacs before it. Emacs has Org-mode. Why shouldn't VS Code have a killer app of its own?
BlahGod420
I (respectfully) disagree completely. All source code is knowledge. All the content that goes in to a product when you're shipping it is knowledge. Managing a project is knowledge management. Managing multiple projects is knowledge management. Managing a company is knowledge management.
I've never seen a tool (other than, arguably, an Operating System) that does all of this comfortably.
hackeredje
When you have a family. A wife who barely knows how to create bookmarks in a browser and kids who only use their phone you need something extremely simple.
So you need to " scale " from 1 person to a household and take into account that all information you put in there, financials, recipes, addresses, etc. etc.. need to be ubersimple, working on a phone, auto sync etc... and need to continue working even when something would happens to yourself.
So that is why i standardized the knowledge management system in our household on OneNote (2016 client for the laptop).
This is because everyone understands office.
The API both the COM and the graph API also lets you create , update, read, etc... so you can write basically everything you can think of e.g. auto put bank transaction on the correct place in the correct table in the correct place in the taxonomy.
Thinking about the structure of a family and everything in there is also interesting and I had some larger revisions on that.
See also: https://github.com/projectje/OneNoteExporterAkaPublisher
hairofadog
This resonates with me. People like to talk about simplicity, and I get that, but to some people simplicity means “just use org mode”, which is a whole different axis of simplicity from “my partner can use it without redirecting their tech-related-frustration anger at me”.
There’s yet another axis of complexity/simplicity, which is that trying all these different note-taking systems means any time I want to look up a note, I have to check half a dozen different places. Recently I’ve been dropping everything except for code snippets into apple notes, and I’ve had a much easier time finding things later.
zie
I use Fossil-SCM for that. 100% OSS, it's a single Sqlite DB file for backups, etc. It's totally fine being offline or online and totally fine being de-synchronized for years and catching up.
I put my legal and financial info on a USB stick with the fossil binaries every year, and hand them out to next of kin. Whenever I die they just plug in the USB and click the appropriate fossil binary and bam, they get everything with a web interface, files and wiki with instructions included.
joshspankit
I’ve always shied away from OneNote, finding it cluttered and as overwhelming as someone else’s heavily dogeared and bookmarked notebook, but you have my attention.
Do you have any breakdowns on how you conquered WAF and the API?
hackeredje
COM interface: https://docs.microsoft.com/en-us/office/client-developer/one...
So just $whatever = New-Object -ComObject OneNote.Application and then lookat $whatever |get-member to see what you can do for members and properties.
---
I tried several things. I needed it because the amount of stuff around a family is just too much to keep in my brain especially with kids who needing a lot of help with their school and trying to also put some fun in it like " nice netflix series you want to share" but also serious stuff like " households tasks, bedtimes etc"
I think I could write a book on what i put in there.
Budget is also interesting, after 20 years I decide on the following structure:
A = income B = budget per household member (so 4 posts) C = everything that needs to be discussed (vacation budget, savings, subscriptions, stuff we want to buy this year, etc) D = everything that we do not need to discuss (mortgage, food, all kinds of city taxes, healthcare etc)
That makes it really simple: only the points in C are needed to discuss "want to spend 20.000 on vacation Or want to spend 20.000 in savings" .
Because I noticed that mosts of the costs stay the same.
Many of these "topics" then tie in or have an overlap with the knowledge management topics from another perspective. So e.g. the budget post C.Subscription.XboxLiveGold ties into culture_and_free_time.games and where we just have fun pages on various game consoles and tips for games we play or we want to share with each other or " favorite games" .
hackeredje
What it also means is that when the thing grows at a certain moment you see a parallel in the taxonomy you put in your local organisation with external " all knowledge ever" for instance on wikipedia portal > categories etc or any of the " all knowledge" systems like dewey or libray systems.
And in many cases you write the thing that has a linkage to already being in a place in other generic taxonomies e.g. when i my son has an exam on e.g. French Grammer we put some stuff in there to undertand these lessons, but this " structure " is also already on wikipedia. But the structure on wikipedia is just too much too handle. So I think that the end-situation for larger parts of the personal household knowledge management system overlaps with the structure found in " all knowledge ever" systems. So im in the phase that I try to look at the taxonomy of these systems e.g. wikipedia and also e.g .structure "mathemetics in the taxonomy that is on the wikipedia portal page for mathematics" (but slower since i only need what is in there what my kid has currently on school). The younger kid can then leverage on this and read up on this. Since we already made it for the older kid.
But I feel that exchanges between "external taxonomies" and " household taxonomies" have not yet been establised. E.g. I also have all my " stuff" in there like " household applicances" (warranties, but also comparison sheets, instructions etc) but the overal taxonomy and the " wording" is matchable with e.g. shops who sell household appliances. If there would be a taxonomy standardization on all of these things it would enable exchange information on meta level.
leeman2016
OneNote is feature rich (specially the availability in iOS) ... but what eventually made me drop it was the unavailability of a reliable desktop app in Linux
joshspankit
Can I just say that this is a space I’ve been deeply interested in for a long long time (I have a terrible memory, and latched on to tech early), and it blows my mind that very few (if any) of these solutions use a graph database as the back-end.
I understand the argument that it’s likely overkill for <10000 entries, but for me it’s been wildly freeing. Especially in cases where the abstraction layer gets out of the way and allows for free-form connections between items. No more rigid db schemas, primary keys, or trying to cram “one or many” connections in to a column.
Shoutout to Neo4j and APOC, as well as to one of my inspirations: https://neo4j.com/blog/building-graph-history-codex/
fphhotchips
I think I come at this problem the opposite way to everyone that actually builds these products/tools. I want to mindmap and brainstorm all of the topics, and then take notes on them - preferably with a time component to that. Even more than that, I want to be able to take notes on the relationships between nodes, since that's where the real gold tends to be.
Mindmap tools generally don't work for this because they want a single topic for each map, and all the words on the one page. KM/KB tools don't work because I can't just draw the relationships between the points, I have to go in and actually create notes that I then get to visualise.
I really wanted to like Obsidian, but the way it treats folders puts me right off. I tried Neo4j Bloom, but there are UI difficulties and it's not really great for notes. I'd build it myself but I have no idea where I'd even start on a UI.
cik
I'm the same way. Mindmaps are by far the best tooling for how I think. They're also just not portable. I keep trying. It would be awesome if mind maps could be united with a console somehow, so that I could inject things from "wherever".
I've spent a lot of time caring about KM/KB and can't settle on anything that survives more than 2 years. What do you do today?
joshspankit
I once built a mind map with a “MUSH” interface (text-based, where you traverse the map “on ground level” as a character). You avoid information overload by showing the synopsis as you “enter” a node, then you could say `e[xamine] here` to view the whole text.
Traversal is quicker than you’d think, and it would likely work well on mobile devices.
It was years ago, but I still think about making that in to something “real”.
fphhotchips
I keep everything in a combination of OneNote and Google Sheets, which I link back to Salesforce with the Salesforce connector for Google Sheets. It is... not perfect, but spreadsheets were the only way I could find to quickly get the information down that I wanted to.
qudat
A lot of solutions referenced in this thread are using graph databases.
It's actually shocking how crowded this space has become.
Everyone is racing to monetize note taking apps and as a result developers are trying to give it away for free now.
fastball
I actually considered using a graph database for our highly connected notes app[1], but in the end went with PostgreSQL because I was following the advice of "go with the technology you know".
It would definitely be interesting to go back and try something like Neo4j, but we've actually managed to wrangle postgres for our graph while still maintaining pretty good performance.
robertlagrant
You still have schemas, they're just in your code.
joshspankit
Agreed, but it’s the flexibility of graph nodes that I’m really talking about.
chrisweekly
yes! roamresearch is backed by a proper graph too
gklitt
Something I find intriguing about Foam is that it's mainly a recommendation for how to stitch together existing tools into a nice integrated workflow, rather than a new piece of software built from scratch.
I wrote a blog post about this idea of "Software as Curation", and how we might imagine more software being built in this way:
https://www.geoffreylitt.com/2020/07/19/tools-over-apps-for-...
jph
Foam is a great FOSS clone of Roam. If you like to write text files, like markdown, and like to link among the text, then definitely try Foam and Roam-- they are superb for organization. And if you use emacs, the similar tool is org-roam for emacs org mode.
toadi
and there is also dendron....
I used all of them and settled on dendron. Because Dendron makes the same links and nodes as roam and all the others. But it is still also hierarchal. After so many years of hierarchy I can't let it go so this hybrid is perfect for me.
cjlm
There is also logseq.
escherize
I am a logseq user. The nice things about it include:
- all your usual backlinks, graph views, etc.
- saves files as plaintext
- md or org mode files
- operates on / syncs / saves plaintext files
- edit locally, or sync with a git repo
- accessible from mobile (via your github repo)
So I use this with org-mode files, and on my laptop I edit files locally (I have it pointed at a git repo), which is very reliable. Then the phone simply reads from (and writes to) the same git repo.
Since it runs on git, the interoperability of the notes is really nice. If I edit the same page on 2 machines or delete a file, it's just a simple merge conflict that I can use my favorite git merge tool to fix.
Not to mention the developer is extremely active and helpful.
JosephHatfield
There is also Obsidian
max_dev
I customized a jekyll template[1] to publish my Obsidian knowledge base and increase the discoverability of notes, which could be better in the app. Also, I wasn't comfortable being tied to the "publish" feature that is still in development.
I use it daily[2].
[1] notenote.link template : https://github.com/Maxence-L/notenote.link
jasonmotylinski
There is also Zettlr.
khimaros
open source and self-hostable: https://github.com/BoostIO/BoostNote.next
hosted version is freemium on web or desktop: https://boostnote.io/
notagoodidea
There is also neuron.
fsnowdin
Obligatory mention of org-mode: There is also org-roam. https://www.orgroam.com/
ydant
I just recently started trying to get my note-taking together and build my own personal knowledge base (or at least stop trying to remember everything and instead store it somewhere to refer back to).
After trying a handful of the usually mentioned tools, I've been very happy with Trilium Notes [1], which seems to share some of the same concepts as this project, but providing rich-text (including embedded images).
As much as I like the flexibility that flat files, markdown, or Org-mode provide, I also really like the extra built-in functionality that comes along with Trilium.
I'm not wild about having the data tied into a single program, but the data is at least accessible and documented (if primarily through the open source code), so I know my data's not entirely locked up.
Trilium says macOS isn't supported as the author doesn't have a Mac, but it works 95% like you'd expect with a few small quirks.
calebkaiser
I posted this in a reply to another comment, but in case it gets buried, as of last month Foam is also published to OpenVSX, meaning you can run it on VSCodium if VS Code's telemetry and licensing give you pause:
ipiz0618
To people who maintain a personal knowledge base: how often do you check older entries of it? And how useful is the graph view as compared to traditional list view note taking grouped by tags?
lervag
My base has somewhere around 200k lines of text. I'm pretty sure some of these are almost never read. But I do open old entries/pages some times. I see no problem in writing a note that I never read. The writing is itself useful, in fact, as it helps me learn and organize information.
(FYI: I use a personal wiki based on Vim (with wiki.vim [0]) and a markdown flavor.)
ipiz0618
Cool plug-in!
jjude
I take notes in an atomic way (other than clipping interesting articles). Because they are atomic, many of the notes are linked in one manner or the other (either via "concepts", "daily log" etc.). So whenever I revisit one of the "concepts" node, I visit these atomic notes.
Also, obsidian has a "open a random note" button. Thus I open a random note once in a while to see if I can learn (or relearn) something + to see if that note is still relevant for me.
ipiz0618
The "random note" function sounds really neat
ricg
A data point: 11 years of note taking → 1,860 notes.
I've been keeping notes for work in a structured way for quite some time.
I log everything that I might want to recall later (project/app ideas, active projects, meeting minutes, phone call logs, people/companies I interacted with).
Typically I use two methods for retrieval:
(1) full text search: I simply search for a keyword; most common access
(2) via backlinks: mostly for people when I need to figure out when I last talked to a person and what we agreed upon. I link the person in each meeting minute so that I get a list for links on the person's page.
Frequency of access:
- For active projects, pretty much each (work) day
- For older notes, infrequently, maybe once a month. The older a note becomes, the less likely it is that I will have to access it.
I started out with Circus Ponies Notebook, migrated to Scrivener, and two years ago started working on my own app, Life Notes[0] (I really wanted links and backlinks and the ability to keep regular files/PDFs next to my notes).
Thorentis
I want a FOSS knowledge management application that supports quick-linking, backlink browsing, a graph visualisation, and tagging. I'm yet to find something that doesn't leave one of these features out, or has some other deal breaker (not FOSS, can't backup, no markdown support, not cross platform, etc).
This looks like it might fill that void, but I don't like that it's based on tying together VSCode extensions. I want something standalone, dependable, and usable purely off-line if need be.
escherize
I believe logseq.com fulfills those needs. Using the "properties" feature, you can tag any block or page.
What's more you can even write queries [0] to ask interesting questions about your notes, like what TODOS were written within a time period, etc.
Could you explain quick-linking a little more?
Thorentis
By quicklinking I mean you can quickly link between pages, usually by using some kind of keyboard shortcut or shorthand. e.g. The [[ in Confluence has become common in most tools. (You type double square bracket and sometimes it has a prompt with recent pages, or you just type the page name and close with two more closing square brackets).
npunt
Neat, but fails in the same way Obsidian et al do: files are not a good abstraction for atoms of knowledge. What if knowledge isn't textual, or is only partially textual?
dsissitka
Obsidian added Roam style block references a while ago.
spiffytech
File-based knowledge systems also assume knowledge is neatly partitionable in relatively large chunks.
y2bd
Notion [0] and recently released Craft [1] are both good personal-wiki applications that support lots of content types directly (from images and videos, to even things like tables and kanban-style boards for Notion), and have good mobile solutions.
Unfortunately, both are proprietary solutions that require paid subscriptions for most of their features and their own special syncing software. Craft also only runs on Apple devices at the moment (they say they're working on a web app).
eddm
For what is worth, Notion has a very generous free tier that makes it easy to try it out and see if it works for you.
- t. Happy Notion user and subscriber.
notadev
Microsoft OneNote has pretty good OCR of documents and full-text search.
nika1975
Unfortunately OCR is in my experience not as smooth as within Evernote or Google Keep. When I take a picture using the OneNote app on Android, OCR never works. You have to use the separate Office Lens App to upload an image and move it to the correct place in your note later. Also: I've got quite a few old PDF docs containing scans of contracts as images. Google or Evernote enablefull text search everywhere, OneNote cannot search for text in these PDFs files .
hu3
I'd love to be able to paste an image and have it figure out the image markup and storage in the directory.
neurotrace
There's a plugin for that https://marketplace.visualstudio.com/items?itemName=mushan.v...
sean_pedersen
Interesting, I just realized I should market my project as a personal knowledge management system as well. Thanks a bunch!
teitoklien
This looks interesting ! Great project !
sean_pedersen
Glad you like it. Feel free to give some feedback, once you tried it out.
ibn_khaldun
Notenik[0] is a FOSS native Mac app for note taking that I have benefited from tremendously.
It has a few UX quirks to it, but the developer is highly active, responsive and friendly. Its strengths lie in its scripting and templating features (check out the site for more info about them) that turn the app into something like a static site generator with a GUI.
[0]: https://notenik.net
Get the top HN stories in your inbox every day.
I believe that knowledge management tied to a development platform is a bad idea. Development isn't the only thing I do in computers and knowledge management should be able to store stuff from all other activities (including cooking recipes or reading notes to read on a mobile phone).
I don't love tiddlywiki but haven't found a better alternative yet. I'd love something easy to configure but powerful like the wiki and bug tracker on the Fossil SCM, but not tied to a specific platform. Also, Markdown/text/html because no vendor should own my thoughts. Obsidian[3] and Notion are in my "to check" list.
[1] https://tiddlywiki.com/
[2] https://fossil-scm.org/home/doc/trunk/www/index.wiki
[3] https://obsidian.md/