Diff-pdf: tool to visually compare two PDFs

Daily Digest email

Get the top HN stories in your inbox every day.

simonw

This inspired me to have Claude 3.5 Sonnet knock out a quick web page prototype for me, using PDF.js to load and render the PDFs to canvas elements and then display visual diffs between their pages.

Two prompts:

    Build a tool where I can drag and drop on two PDF files and
    it uses PDF.js to turn each of their pages into canvas
    elements and then displays those pages side by side with a
    third image that highlights any differences between them, if
    any differences exist

    rewrite that code to not use React at all

Here's the result: https://tools.simonwillison.net/compare-pdfs

It actually works quite well! Screenshot here: https://gist.github.com/simonw/9d7cbe02d448812f48070e7de13a5...

radicality

What’s the best way to effectively make use of Claude 3.5 ? I signed up a few days ago for the api access. Besides console.anthropic.com , do you recommend any other tools that I can run locally to give it api key and use claude effectively ?

simonw

The web interface gives you access to the Artifacts feature where it can build SPAs and render them in the browser.

For terminal access I like using my own https://llm.datasette.io/ tool with the https://github.com/simonw/llm-claude-3 plugin

For Python library access I recommend checking out Claudette: https://www.answer.ai/posts/2024-06-21-claudette.html

kenjackson

How much additional work did you have to do to get it into this form? Thats impressive work using those prompts.

simonw

Almost no extra work at all. You can see the full transcript here: https://gist.github.com/simonw/9d7cbe02d448812f48070e7de13a5... - it really was just those two prompts, then I copied the result out into a document to test it.

I modified the HTML a tiny bit before publishing it - I set the font to Helvetica and added the note at the bottom of the page showing the prompt I used.

The whole project took less than 5 minutes - then another 10 to write it up.

robertlagrant

That's incredible.

hallucinated

It's hard to challenge a comment that is 4 days old: What's the difference between the 2 PDFs?

I didn't find any actual difference. But - Maybe it's just me that's hallucinating

undefined

[deleted]

tomwheeler

In a previous job, I had to validate the output of an unreliable production publishing system, so I tested dozens of PDF comparison tools available at the time. The best I found was called Delta Walker. It was proprietary commercial Mac-only software, but reasonably inexpensive, accurate, and could handle long PDFs with lots of graphics well.

I remember evaluating this diff-pdf tool and finding that it fell short in some way, although it's been so long that I don't recall the specifics. Most of them failed to identify changes or reported false positives. I also remember being disappointed since this one was open source and could easily be scripted.

pivo

It looks like Delta Walker's added Windows and Linux support: https://www.deltawalker.com/download

Always42

I found "Draftable" to be great

mksreddy

Wouldn’t exporting pages to images and using pixel diff accurately identify differences in PDF’s?

netsharc

I guess it depends on the use case. Imagine adding an extra sentence in the second PDF, and this causes the paragraph to have 6 instead of 5 lines, and the next paragraph begins a line further down, and the last paragraph of that page ends up in the next page, etc...

mksreddy

Thanks. That helped understand it better.

ydant

Related - this might be helpful to someone.

ImageMagick can do a visual PDF compare:

    magick compare -density "$DENSITY" -background white "$1[0]" "$2[0]" "$TMP"

(density = 100, $1 and $2 are the filenames to compare, $TMP the output file)

You need to do some work to support multiple pages, so I use this script:

https://gist.github.com/mbafford/7e6f3bef20fc220f68e467589bb...

This also uses `imgcat` to show the difference directly in the terminal.

You can also use ImageMagick get a perceptual hash difference using something like:

    convert -metric phash "$1" null: "$2" -compose Difference -layers composite -format '%[fx:mean]\n' info:

I use the fact you can configure git to use custom diff tools and take advantage of this with the following in my .gitconfig:

    [diff "pdf"]
        command = ~/bin/git-diff-pdf

And in my .gitattributes I enable the above with:

    *.pdf binary diff=pdf

~/bin/git-diff-pdf does a diff of the output of `pdftotext -layout` (from poppler) and also runs pdf-compare-phash.

To use this custom diff with `git show`, you need to add an extra argument (`git show --ext-diff`), but it uses it automatically if running `git diff`.

bigfatfrock

Next level, especially with the git attribute calls, well played.

I'm still blown away how powerful imagemagick is after using it for a decade or two, what an inspiring piece of open source software.

Bluestein

imagemagick really is magical.-

thibaut_barrere

I have been using this in a CI pipeline to maintain a business-critical PDF generation (healthcare) app (started circa 2010 I think), here is the RSpec helpers I'm using:

https://gist.github.com/thbar/d1ce2afef68bf6089aeae8d9ddc05d...

The code contains git-stored reference PDFs, and the test suite re-generate them and assert that nothing has changed.

Helped a lot to audit visual changes, or PDF library upgrades!

tylerflick

Are you using singed digests in the PDFs?

pmarreck

could you not just compare the source (or perhaps even the hash) of the PDF and assert on that?

ydant

I use some custom tools for PDF comparison (visual, textual, and perceptual hash) for my personal records/accounting purposes.

A number of the financial and medical institutions I deal with re-generate PDFs every time you request them, but the content is 99-100% identical. Sometimes just a date changes. So I use a perceptual hash and content comparison to automate detecting truly new documents vs. ones that are only slightly changed.

jabroni_salad

If the document is a legally required disclosure (like a bank's fee schedule for example) then you need to grade that document directly rather than its source code. PDFs are horrible and there is a lot that can go wrong with making them between writing and publishing.

alexdoesh

Hashes can change regularly due to metadata. Source checks may also require some filtration or preprocessing before comparison. Visual comparison is the best option here, especially if you have a complex document with multiple third-party components that may change both the hash and source but keep the visual appearance the same.

thibaut_barrere

In this case, we indeed have multiple components (although not third-party), and being able to refactor those without risk is quite nice.

knallfrosch

What should I do when the assertion fails – inspect the PDF with my sad little caveman eyeball?

ggrosskopf

In my own tests where I inspect PDF differences in python, I iterate through the pages, if the number of pages is the same, I convert each of them with PIL to bitmap, get the diff (ImageChops.difference is black for everything same and colored for diffs) and find the content of the diff with `getbbox`. This gives me the coordinates of the rectangle where changes appeared, I then use those to also print the page with a colored rectangle and print out the crops.

I give out the original page, the original rectangle, the original page with colored rectangle, the new page and the new rectangle, the diff cropped and uncropped only after which I start using my caveman eyeballs

I also pixelate it a bit and have a brightness cutoff for the diff to see if the diff actually matters and i also try if re-cropping a bit so shifting by a limited amount of pixels makes it look like an ignorable difference because everything just moved to the left a bit but that is optional.

I also recommend exporting the new pdf from the CI/CD tool to be put back into the test as reference. Even between Linux distros and versions small changes in fonts and stuff like that make a difference

thibaut_barrere

the source sometimes changed for small internal reasons in the library generating the PDF (prawn). So just comparing the source would not give a clear cut answer. A visual comparison has helped quite nicely over time.

poidos

Reminds me of the tool Bob Nystrom wrote to help himself out when working on the physical edition of Crafting Interpreters: https://journal.stuffwithstuff.com/2020/04/05/crafting-craft...

Whole article is worth reading, but if you want the relevant bits search for “ I wrote a Dart script that would take a PDF of the book”.

jaustin

We've been using this in the Micro:bit Educational Foundation (microbit.org) to fill a gap in hardware design tooling, and get visual diffs of our schematics and gerbers during PCB design iterations. It's kinda wild that's what we ended up doing, but if you want to be sure your radio layout didn't change at all when you're making a minor revision to a different part of the board, visual diffs are perfect.

That said, next project we want to try something more integrated with EDA tools. If anyone else has followed this path, we'd love to know.

mikeyinternews

You can do this with Beyond Compare (it's not free, but not very expensive either) https://www.scootersoftware.com/

Rinzler89

Beyond Compare is one of those priceless tools I pay for myself instead of waiting for my employer to pay for it. Price/functionality wise it's worth its weight in gold, it's cross platform, and its licensing is very liberal. There's just no FOSS compare tools out there that can match BC.

hipnoizz

What are BC features that you find to be so great?

I'm genuinely curious - I heard of lot of BC being 'the tool' for diffing. I'm used to Meld, but my current employee has a pretty strict policy which tools could be used so at some point I've managed a licence for some older version of BC. But for some reason I've found its UI/the way it works a bit less optimal that I was accustomed for. Since I'm using that primarily for text diffs these day I usually use a diff tool from IntelliJ Idea (I have Idea open all the time).

netol

In comparison, Meld is not stable, nor fast, especially for big diffs. The UI is also more limited. Araxis Merge and WinMerge are good alternatives

smartmic

I like this tool better: https://www.qtrac.eu/diffpdf.html

It shows the differences in the GUI side-by-side instead of overlayed.

Tryk

From the github:

Another option is to compare the two files visually in a simple GUI, using the --view argument:

$ diff-pdf --view a.pdf b.pdf

This opens a window that lets you view the files' pages and zoom in on details. It is also possible to shift the two pages relatively to each other using Ctrl-arrows (Cmd-arrows on MacOS). This is useful for identifying translation-only differences.

yencabulator

Shifting the offset is very far from the experience of a side-by-side diff, and more useful for nudging the images to align them.

justinnk

There is also an open-source/free version of this [1], which I use regularly. You can install it, e.g., in Fedora, with the ‚diffpdf’ package. It is no longer maintained but works very well, has a nice GUI with a side-by-side view, drag&drop support, and both text and visual modes.

[1] https://www.qtrac.eu/diffpdf-foss.html

invalidlogin

I use BeyondCompare 5 for this.

rawbert

We use this tool in our team regularly for comparison of PDFs we obtain from third party services that might have changed after code-changes on our side. Big thanks to the author <3

canistel

Interestingly, Github thinks the project is 46% shell, due to the fairly huge wxwin.m4.

infecto

I noticed this a while back with a private project of mine. The Github languages breakdown seems broken. Mine is a Python project with a handful of Jupyter notebooks but many many python files. The LOC must be 80% python files but Github sees the project as 50% Jupyter.

badlibrarian

You can tweak/exclude with .gitattributes

https://github.com/github-linguist/linguist/blob/master/docs...

infecto

I had no idea. Thanks for sharing.

deckar01

I wrote a pixel-based visual diffing algorithm long ago that was intended for a CI tool that finds all of the UI changes in a PR. I broke the layout of a page I didn’t even know existed as an intern at Inkling and have had this idea in my head ever since.

https://github.com/deckar01/narcis

crocal

I will just chime in to mention Draftable (https://www.draftable.com/compare). It really works well. It’s not so easy to have a visually comfortable diff of two PDFs.

Daily Digest email

Get the top HN stories in your inbox every day.