Get the top HN stories in your inbox every day.
AceJohnny2
graphviz
When we started Graphviz, we vaguely sensed the potential for graph databases or environments, like neo4j, but we didn't work much in that area. It would take more resources, or intensity, than what we had. Anyway, gvpr is a modest attempt at a sort of graph query language.
Neo4J is interesting. It does something important, mainly, rationalize the mess of graph representations and algorithms. It's not hard to understand why that is valuable, but Neo4J sell it as being "efficient for linked structures" (relational databases don't have foreign keys and indices) and "intuitive" (have you ever tried to read the code for graph matching in Cymbal? I rest my case.) They have investments of hundreds of millions? Worth billions? They contributed something valuable to the tech economy.
It's surprising the conventional relational database products don't pay much attention to this opportunity, but they have gone vertical, e.g. credit card processing, because in the end it's more profitable to solve big end customer problems.
AceJohnny2
I'm actually disappointed that `gvpr` isn't mentioned at all on GraphViz's main page, because I think it's its most powerful tool. All I could find are manpages, and not even on GraphViz's own website!
pvaldes
Never had heard about it and use graphviz since... dunno, more than 15 years probably.
sterlinm
Are there any good examples of how you'd actually use this in practice?
j0057
I don't know of any example, but I immediately thought of a use case: to use it as a sort of CSS language where you take a graph with only 'semantic' custom attributes and use the GVPR language to transform these into shape and style attributes that are recognized by Graphviz, so that you don't have to repeat all the style information for each node and edge. That's already pretty exciting to me!
steveroush
I second the GVPR praise. Pretty powerful.
paphillips
I've used GraphViz a number of times and highly recommend it as a standard tool on your belt. Having a stand-alone executable that can export to SVG is great.
The most complex thing I've done with it [1]: a tool (MIT-license) that builds diagrams of the data and addressing pipeline for a DSP processor, and lets one 'scrub through' the assembler code frame by frame and see the values propagate through the blocks.
Also PlantUML [2] uses it for most diagrams.
Getting layout and positioning the way you want can be tricky but is usually achievable with patience and hidden objects.
[1] https://github.com/paphillips/DFB [2] https://plantuml.com/graphviz-dot
thrwawy283
My graphviz-fu got so much better when I started using invisible objects for the grid-like layouts. At that point you're just using graphviz for positioning/spacing/styling instead of creating the overall topology.
buzzwordninja
This sounds interesting, can you expand on that a bit please?
thrwawy283
You're handing graphviz a list of relationships. Sometimes you know the ideal way to visualize this is a tree, or something with right angles, but graphviz isn't laying it out that neatly. So you create extra nodes, and edges to those nodes, so that graphviz considers it a "balanced" tree or grid. Then you use styling to turn those nodes invisible (showing just the nodes/edges you want). Once I have this correct I usually contain it within a "cluster", and then link that cluster to the larger diagram.
pvaldes
if you want X nodes falling near in the graph you just put them into a box with white lines and they move as one from this point. Can be made with the cluster environment.
pvaldes
You can use invisible clusters also to group things
zackees
[dead]
nyellin
Here are some cool things I've done recently with graphviz:
1. Draw "sketch" style graphs using https://sketchviz.com/new
2. Draw graphs of memory usage in golang programs (using pprof): https://graphviz.org/Gallery/directed/pprof.html
3. Embed graphviz into Sphinx docs (shameless self-promotion: https://docs.robusta.dev/master/catalog/triggers/index.html#...)
rajandatta
My single biggest issue with Graphic is that the quality of Arrow end-points is very poor. They don't terminate cleanly at the destination. This is visible in the 2nd link you've posted.
I'm amazed that this hasn't been addressed. It's been there for years.
Makes the end products impracticable for distribution in a professional setting. It's Ok if you're the only consumer ... not great for other cases.
feanaro
I remember this issue being raised, I think on the graphviz Discourse, and the graphviz developers responding to it as if it was a hard to solve limitation of graphviz for some reason. But I can't for the life of me remember the details.
graphviz
"Very poor" seems harsh, but, yes, it hurts us, too. (That, and text with slightly-off baselines.)
The relevant code starts around https://gitlab.com/graphviz/graphviz/-/blob/main/lib/common/... If this isn't right, maybe somebody can figure out why. Perhaps the loop stops sometimes on the wrong side of the boundary? Anyway, it's equally possible that when the endpoint coord is handed off to a lower level driver, the arrowhead mitering is wrong. There is no question there was once upon time explicit code to try to cope with this problem, at least in the native PSgen, but I can't find it now. Maybe it wore away as waves of open source development washed over it, along with static tables for a bunch of "standard" PS fonts.
For more background, see for example https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/s...
In the worst case, this has to be debugged for all the drivers in https://gitlab.com/graphviz/graphviz/-/tree/main/plugin
No doubt there are more details if anyone can remember them.
pphysch
Ostensibly each arrow style graphic would need an explicitly-defined "attach point" at whatever is deemed the "tip", so that the tip can be oriented perfectly in relation to the vertex and edge of the GraphViz graph.
If nothing like that exists, I can see how it would be a very troublesome refactor.
pvaldes
I never had this problem. Arrow style is configurable if I remember correctly. You can configure (extend) also overlapping in boxes so the arrows never enter into the box area.
I'm talking by heart and I could be wrong in any case.
IAmEveryone
I once created a dataset of X-hooked-up-with-Y for my circle of friends. About 50 nodes and a few hundred connections (after I asked around a bit for additions).
It became a wedding present for one rather central "node" with the title "drosophila neural regulation network" or something like it. No names, just lines and circles. It's still the centerpiece in his home office.
erwincoumans
Very cool. About Sketchviz
1) is there an option to make the graph horizontal instead of vertical? 2) can you make the svg option separate, instead of hidden behind the PNG button (with CTRL key)? 3) how did you do the handwriting style?
hun-nemethpeter
Yes, the name of this attribute is rankdir.
See in the docs: https://graphviz.org/docs/attrs/rankdir/
In the sketch example:
digraph G {
rankdir="LR";
graph [fontname = "Handlee"];
...taubek
I like the output of Sketchviz. Do yo know if it is possible do make such output directly from desktop app?
fsiefken
you can install it locally with docker https://github.com/iamacarpet/sketchviz-docker
taubek
Thanks! Docker makes my computer so slow... And it would probably be an over kill for me. But it is nice to know that it exists.
chuchana
Haven't tried it yet, but this looks pretty spot on: https://github.com/gpotter2/sketchviz
erwincoumans
Sketchviz look great. Is there an option to export vector graphics (for example svg)?
cldellow
If you hold down control, the "Download PNG" button becomes "Download SVG".
Super intuitive, I know! :)
erwincoumans
Haha, why is that feature hidden? Like the latest Tesla Model 3 v11 update hides many features :-)
mbostock
I love Graphviz! It’s in the Observable standard library as the dot tagged template literal, powered by Viz.js.
https://observablehq.com/@observablehq/dot
Here’s a recent example where I used it to figure out why certain dependencies are in my node_modules:
rgovostes
That said, compared to the visual clarity of most of the visualizations you share, the graphs that come out of Graphviz are pretty hard to read. If you zoom in and pan around you can easily get disoriented in some patch of nothing but spaghetti lines. This is definitely an area where some mbostock magic would be of huge benefit.
jahewson
Is there a fork of viz.js that’s still maintained? The original was abandoned in 2018.
JohnHaugeland
no.
two steps above the abandoned version, before the web threads attempt, works quite nicely.
funstuff007
two commits previous or two releases previous to final version?
motohagiography
Graphviz is an amazing tool, it has just suffered from being cloistered to technologists whose jobs aren't to solve the kinds of problems it is specifically adapted for. I use graphs in security and privacy quite a bit, and even built the tech for a hopeful security platform using a graph back end. They yield the fastest path through complex problems, and I use them to do in a couple of hours what typically would take client staff months.
They're really compelling but oddly unpopular.
Imo, they're useful for one-off pattern discovery, and they're most valuable for finding single or a few exceptions and outliers in normalized data, and you need to be in an environment where there are asymmetric returns on finding those. Surveillance seems to be their default use case, with open ended scientific research a close second. This comes up with graph based recommendation engines, which are essentially a surveillance/marketing product based on preferences. Most businesses aren't based on discovery of anything other than customers for a transaction they already have.
These aren't problems engineers typically solve, which are more about scaling and optimizing, they're more marketing and sales problems, where you're looking for exceptions and opportunities. (security and privacy are the complementary antithesis of these) A graph based product (imo) is ideal for product marketing analysts optimizing for customer preferences and discovery.
From a product perspective, graphs are analogous to ML, where you'd use a clustering algorithm on loosely structured data to yield categories, comparisons, and implied relationships, whereas a graph yields the same thing over the structured normalized data you can feed into it once you have imagined an ontology to fit it.
lizen_one
That sounds really interesting! Can you give a few more details how you use Graphviz? How does it give such a great advantage "to do in a couple of hours what typically would take client staff months."
Did I understand it correctly that you use it to discover patterns? Are these patterns discovered by just using the layout engines? Arent' other tools, e.g. networkx in Python or cytoscape in javascript easier to use interactively in a REPL? What is the typical workflow (maybe plot, find interesting pattern, change query/data in a loop)?
I'm really interested in how Graphviz can be so great. I am currently working with the other mentioned tools for visualization purposes.
motohagiography
Really simply, I typically use Neo4j, but if I have flat homegenous data, I just use Graphviz because the dot markup format lends itself to parsing easily in awk command lines. The times I have used networkx was when I needed a graph abstraction layer to reason about another graph query, so networkx wasn't used as a persistent graph store, but more of an intermediate data structure for orchestrating multiple service and api calls, like a low rent graphql. I'm a crap developer, but the graphs were what i needed to piece the logic together coherently.
One example of clients taking months is mapping counterparties to agreements. Let's say you have inherited a division that has file share full of contracts and you want to understand the line of business. You get the counterparties out of the contracts and find all the paths for obligations between entities within the division and their counterparties. The graphviz/dot layout gives you a map of all those parties in a single slide and shows clusters, instead of a 3 lb. document with a paragraph for each of them that would have cost a massive amount of consulting time, or interviewing several people to get their narrative understanding of how the business worked, the graph provides an objective map. You could just use D3js, but for me the dot markup was faster on the command line than structuring json.
The idea is if you can formulate a conceptual, narrative ontology of an organization, you can create a grammar of things and relationships, and then you can plug data (contract counterparties) into that model and form a fairly complete map.
Another recent use case was enterprise vulnerability scan data over a very full /16 address block, allocated across multiple divisions under different management hierarchies with thousands of hosts. By linking the host ownership data to projects and an org chart with the types of vulnerabilities, I could demonstrate in a couple of slides what the highest impact patching strategy would be. Again, graphviz for sketching up the ontology, then Neo to do the lifting.
On a much simpler scale that was more graphviz/dot oriented, I did some work for a startup where I worked with the executive team who had acquired a codebase and talent, and created an ontology of their pipeline customers, their stated needs, implied product features, platform dependencies, our service interfaces, their code bases, and demonstrated the flow of how work on the code bases flowed through to impact revenue. This ultimately got represented as a Sankey diagram, but it was graphviz/dot I used to sketch up the initial ontology.
jcims
Have you tried gephi? It's not exactly an alternative to graphviz (eg. you can't cluster nodes) but it handles much larger graphs and has a bit more flexibility in layout. It has plugins for both dot and neo4j input.
Graphs and infosec go very well together.
SavantIdiot
I found the learning curve to be quite steep with few examples of graphs that looked well-designed. By default, large graphs look like crap because engineers designed it, not designers (it takes both!).
However, that was back in the late 90's. Now the internet has many examples of better looking layouts, but it is still disappointing that they don't look ... designed?
But integration and automation are great. Like GNUPlot, which I've used for decades because it is so easy to automate once you learn how to make plots look better than the default.
GStreamer uses DOT output by default. But any reasonably sized pipeline is almost impossible to read without excessive zooming. But it gets the job done.
enter-haken
I use graphviz to get a quick postgresql schema overview (good for small to medium databases)
ccakes
Autogenerated database documentation is often pretty hit and miss but tbls[1] does a pretty good job in that space. Especially when you comment on your tables, fields, views, functions etc (which is a good habit anyway!) the output is quite useful
magicpointer
In a similar vein there is Schemaspy[1]. It generates a static documentation website for your DB, which also uses GraphViz to build ER diagrams.
kh1
That looks really cool. Would you possibly share the script?
enter-haken
It's part of the repo.
https://github.com/enter-haken/schema/blob/master/schema.sh
It's a mixture of sed, awk and sql.
kh1
Oh I did take a quick look at the repo but didn't notice it was in the SQL. Thank you!
zoomablemind
Another graphing tool is Pikchr (https://pikchr.org) from the creator of SQLite; actually the SQLite doc SQL-statements diagrams were (re)done using the Pikchr. It's some kind of extension of PIC language.
I'm not very proficient with graphing, so can't compare it to Graphviz, but a few examples I tried were relatively easy.
ggerganov
For the ASCII lovers, I made this simple web page to help convert dot to text diagrams:
https://dot-to-ascii.ggerganov.com
It's useful to embed diagrams directly in source code for example.
Here is a sample [0]:
0.6
┌─────────────────────────┐
▼ │
┌───┐ 0.8 ┌───┐ 0.1 ┌───┐ 0.7 ┌───┐ 0.2 ┌───┐
│ d │ ─────▶ │ │ ◀───── │ e │ ─────▶ │ a │ ─────▶ │ b │
└───┘ │ c │ └───┘ └───┘ └───┘
│ │ 0.4 │
│ │ ◀──────────────────────────────────┘
└───┘
┌─────────────┐
│ @tweet2doom │
│ │
│ ROOT │ ◀─────┐
└─────────────┘ │
▲ │
│ │
│ │
┌─────────────┐ ┌─────────────────┐
│ @player1 │ │ @player2 │
│ │ │ │
│ /play 50-u │ │ /play 30-l,50-f │
└─────────────┘ └─────────────────┘
▲ ▲
│ │
│ │
┌─────────────┐ ┌─────────────────┐
│ @tweet2doom │ │ @tweet2doom │
│ │ │ │
│ Video │ │ Video │
│ │ │ │
│ Start: 1 │ │ Start: 1 │
│ End:50 │ │ End:80 │
└─────────────┘ └─────────────────┘
▲
│
│
┌─────────────┐ ┌─────────────────┐
│ @tweet2doom │ │ │
│ │ │ @player2 │
│ Video │ │ │
│ │ │ /play 30-l │
│ Start: 80 │ │ │
│ End:110 │ ──▶ │ │
└─────────────┘ └─────────────────┘
ccakes
Awesome! Since switching from Mac to Linux I've been really missing Monodraw[1] for generating diagrams suitable for code comments. Emacs artist-mode is nice but I struggled to get the hang of it, will definitely be trying this!
Godel_unicode
If you're on mobile you're going to want to rotate your phone.
benreesman
This is so friggin cool.
I would kill to have this as like a Python library or something, any chance you’d open-source it?
Or is there a GitHub link already and I overlooked it?
Either way great job.
ggerganov
Thanks!
It's already open sourced on Github [0] (there is a link on bottom right of the page). Note that the page is simply a wrapper of Graph::Easy [1], so nothing technically interesting to see in the repo - it just passes the HTML input to a command-line tool and prints the result.
You can either use Graph::Easy directly on the command-line or you can use python to make an HTTP request to my page (example is shown in the README of [0]).
cjlm
I love this and wrote about it in my newsletter on network thinking [0] – thank you!
tptacek
This is fantastic.
karlicoss
Graphviz is awesome!
Here are a couple of my diagrams:
https://beepb00p.xyz/blog-graph.html -- graph of my blog pages with tags/connections between posts, generated with a DSL-ish python script https://github.com/karlicoss/beepb00p/blob/master/misc/index...
https://beepb00p.xyz/myinfra.html -- map of my personal data & infrastructure (discussed a year ago https://news.ycombinator.com/item?id=26269832 ). Also a similar DSL https://github.com/karlicoss/myinfra/blob/master/generate.py
The main downside for me is that sometimes it gets the positioning wrong, and you can see how it can be easily fixed, but it's hard to convince graphviz to actually do so. Basically I'd love a tool where I can do 10% of positioning manually and let the rest be constraint based like in graphviz.
taeric
It used to have a built in tool to move nodes it has placed. All I can find scanning the docs right now is the -n flag to neato, which will honor existing pos fields.
Edit: dotty is what I was trying to remember.
triggercut
Graphviz has saved my bacon so many times throughout my career. Not just for visualising constraints and dependencies in planning and scheduling (construction, not computing) but in so many other areas.
Best example was for fabrication of a large subsea structure. Often, these critical assets have material traceability requirements on-par with space exploration, producing thousands of pages of certificates and documentation along the way.
I.e. you need to prove and assure the provenance of every piece of material sitting on the 1km down on the ocean floor back to the assay report of the soup in the ladle at the forge when it was formed, and everything in between; material certificates, design specs, third party testing, mother plate to material stock to final cut part identification, and all the third-party witnessing and assurance along the way. Easy for small parts but cumbersome for 60tn structures with thousands of parts and welds.
When the requirement is 100% QA/QC coverage, linking IDs to certificates and other documents in each part of the chain meant we could easily visualise things and look for rogue elements like child parts pertaining to be from two seperate mother plates, or destructive testing coverage for all materials.
Not how it's usually done but it was a useful tool for communicating to other not so technical stakeholders and helping some on the project sleep easier. :-)
lwerdna
I use graphviz probably weekly and it's actually become a bit of a productivity hack. If there's something I don't want to do, I can often convince myself to just DRAW it instead, usually with graphviz. Once drawn, the actual task is easier, both because the barrier of starting is overcome, and because the drawing is useful in the task.
For example, I had to parse DWARF debug info recently to scrape type information, something I dreaded doing. Instead of diving into the DWARF specs, I set out to adapt one of the pyelftools [1] examples to produce a .dot file and graphed it with dot, producing [2].
Now looking at the picture, it's nearly obvious how functions and structs and types are stored. The rest is trivia (How do I access this attribute? How do I iterate over a DIE's children?).
Here's a simple alias so that anything that writes to /tmp/tmp.dot can be viewed with a single command:
alias graph='dot -Tpng /tmp/tmp.dot -o /tmp/tmp.png && open /tmp/tmp.png'
edwinyzh
Wonderful, the graph is much clearer than text specs.
dang
Past related threads:
How Graphviz thinks the USA is laid out - https://news.ycombinator.com/item?id=25611053 - Jan 2021 (80 comments)
Create diagrams with code using Graphviz - https://news.ycombinator.com/item?id=23475225 - June 2020 (211 comments)
Graphviz – Graph Visualization Software - https://news.ycombinator.com/item?id=18797932 - Dec 2018 (3 comments)
Show HN: Project Management as Code with Graphviz - https://news.ycombinator.com/item?id=15950325 - Dec 2017 (64 comments)
A Quick Introduction to Graphviz - https://news.ycombinator.com/item?id=15324883 - Sept 2017 (44 comments)
A Technique for Drawing Directed Graphs (1993) [pdf] - https://news.ycombinator.com/item?id=14568180 - June 2017 (11 comments)
What happened to GraphViz? - https://news.ycombinator.com/item?id=11265188 - March 2016 (2 comments)
Ask HN: What is your favorite application of Graphviz? - https://news.ycombinator.com/item?id=790668 - Aug 2009 (8 comments)
There have been others, of course, but these seem to be the ones most focused on Graphviz itself.
Get the top HN stories in your inbox every day.
The `gvpr` tool in graphviz deserves more recognition.
Graphviz, at its core, is a simple graph description language (DOT), which allows progressive enhancement, and a bunch of auto-layout tools where some double as renderers.
The double-duty of the layout/renderers masks GraphViz's true power, which is that you can create a pipeline modifying your graph source (confusingly referred to as '.dot' files), progressively adding attributes.
This is where gvpr comes in. While most are familiar with GV's all-in-one layout+rendering tools like `dot` or `neato`, gvpr is a generic Awk-like programming tool that iterates over nodes and edges. With it, you can implement any layout algorithm you like (or not!), to specify part or all of the attributes of your graph (like positioning and style), to then be rendered using `dot` to the format of your choice.