Get the top HN stories in your inbox every day.
wowi42
zmmmmm
> The honest move is to admit you wanted code, then write code
This war will never end ... because there are genuine tradeoffs on both sides. YAML being a bad data description format isn't actually central to the question of whether you describe infra as data or as code. You can use JSON if you want. Data is static, 100% predicatable. Code is non-deterministic right up to the halting problem. If your infra should look different on wednesday to thursday, well it can do that! Some people like it, some people think it's the definition of hell.
Terraform makes an interesting tradeoff to try and have the best of both worlds but ultimately still falls on the same issue ... I've not seen one project yet of any complexity that didn't use workarounds to implement optional components (let's just pretend there's a list of them and it has 1 or zero elements in it!).
Ultimately I agree with your philosophy but maybe not your language. IMHO You really want a language that is built from the ground up around static typing and immutable constructs for this. Get as close to that predictable determinism as possible. But then, if the whole world knows python, I guess python it is.
crispyambulance
> Infrastructure as Code, not infrastructure as YAML.
Right on.
It's amazing to me that we've spent decades with programming languages and environments which can accurately guess what you're about to type next, which have enormous expressiveness while maintaining cogency, which are intuitive and well understood by humans, which have endless libraries and an infinity of ways of connecting with the world.
And what do we use to configure the most sophisticated infrastructure to run such code? Yet another mark-up language!
beagle3
Many domains are better served by a more limited programming language, so you can analyze a program and/or make guarantees about it.
Real regexes (actually regular…) are infinitely better than Python code matching the same string (if they are sufficient) - you can compute their intersection, union, complement; check if they can match anything at all (and generate an example automaticallly).
For software builds, Bazel and others use Starlark, which is a restricted Python subset, so builds can be guaranteed finite and can be reasoned about.
Ansible may or may not offer any benefits in return for the limits (I am not an ansible guru), but in general, most tasks do not need a Turing complete configuration/specification language - and it is then better to NOT have Turing completeness.
VectorLock
The "you don't want a full programming language" trope I see repeated a lot but I think far more people end up wishing for a Turing complete language than wishing it _wasn't_ Turing complete.
crabbone
> It's amazing to me that we've spent decades with programming languages and environments which can accurately guess what you're about to type next, which have enormous expressiveness
You've almost guessed the problem. Too much expressiveness is a bad thing. This is a problem I encounter a lot more often then I'd be happy to. It's very often is much easier to build something more generic than what the user actually needs, and then testing it becomes a nightmare.
To make this more concrete, here's a case I'm working on right now. Our company provides customers with a tool to manage large amounts of compute resources (in HPC domain). It's possible to run the product on-prem, or in different clouds, or a combination of both. Typically, the management component comes with a PXE boot and unfolds from there. A customer wanted integration with a particular cloud provider that doesn't support this management style, nor can it provide a spare disk to be used for management, nor any other way our management component was prepared to boot.
The solution was to use netboot that would pre-partition the disk and use the first N partitions to store the management component as well as the boot, ESP / bios_grub partition etc. It had to be incorporated into the existing solution that encompasses partitioning and mounting all the resources available to a VM, including managing RAIDs, LVM, DM and so on.
The developers implemented it as a GPT partition name with a pre-defined value that would instruct our code to ignore the partitions found prior to the "special" partition and allow the user to carry on as usual, pretending that the first fraction of the disk simply didn't exist (used by netboot + the management component).
This solved the immediate problem for the user who wanted this ability, but created thousands of problems for QA: what happens if there's a RAID that uses the "hidden" partitions? What happens if the user accidentally creates second /boot partition? What happens if the user wants whole-disk encryption? And so on. It would've been so much better if these questions didn't exist in the first place, than to try to answer them, given the "simple" solution the developers came up with.
If you programmed for just a year, I'm sure you've been in this situation at least a few times already. This is exceedingly common.
* * *
There's an enormous value to being able to restrict the possible ways a program can run. Most GUI projects? -- They don't need infinite loops! It just makes programs unnecessarily hard to verify. But it's "easy" to have a single loop language element that can be made infinite if necessary. Configuration languages exclude whole classes of errors simply by making them impossible to express.
However, I have to agree that, specifically, YAML is a piss-poor configuration language. It has way too many problems that overshadow the benefits it offers. We, collectively, decided to use it because everyone else decided to use it, making it popular... and languages are "natural monopolies". So, one could certainly do better ditching YAML, if they can afford to go unpopular. But ditching the idea of a configuration language is throwing the baby out with the bathwater.
bmurphy1976
Thank you for this. I've implemented my own version of this a couple times over the previous 25 years. This is how my code always looked.
I've used Salt, CFEngine, Chef, Puppet, Make, Bash, and many hand-rolled iterations of this approach. I finally threw in the towel and forced myself to come to terms with Ansible and it's quirks because I needed the wider community support.
Now with AI tooling, I'm not so convinced the community modules moat is an actual moat. I'm going to very seriously consider porting all my Ansible code to this and see how it feels. I anticipate I'll be much happier after the change.
Do you have any plans to integrate with/build on other communities modules? i.e. even if it's not perfect, being able to call Ansible or Salt modules from PyInfra would be one way to fill the gap.
mxey
The problem is that it is actually not just Python, branched with “normal if statements”: https://docs.pyinfra.com/en/3.x/deploy-process.html#checking...
bmurphy1976
You can write `if CHECK: do something`. There's nothing preventing that.
I've been down this path, implemented my own version of PyInfra many times over the years. I've used Ansible and my own implementations in anger. The _if param is far far far from the worst offender and it's a natural addition, especially when you are laying out a bunch of unrelated checks into something that looks more like a table.
Fizzadar
This! Been trying to find the best (least worst) solution to this since 2015 when I started pyinfra. Done ast parsing/hacking, done weird context managers instead, tried rewriting statements to context managers. _if is the latest, and I think least worst, option right now.
Basically a flaw of the entire model where you write code as if executing a single host which is then executed on many in parallel, forcing the two step diff and deploy that causes this.
Funny thing is since v3 this behavior (diff then execute) is even desired with the yes prompt like terraform.
aniceperson
I don't understand why not use function chaining, Javascript style. Make the order of execution explicit through scope.
zahlman
As a heads-up, your comments here were flagged. I think some people must have thought your (current) writing style rather LLM-ish.
mrkstu
It obviously was LLM assisted, but I think collectively we will have to get over our distaste for text that has some LLM’isms in spots as long as it isn’t obviously completely outsourced to a bot, unless we just want to shut down message boards completely.
wowi42
English is not my first language, so I lean on an LLM to clean up the frenchisms, but the ideas are mine :-)
scorpioxy
What was the obvious part in it? I still can't tell.
undefined
wowi42
Ah shit...
mattbillenstein
I should try this myself, but I also rolled my own https://github.com/mattbillenstein/salty
In the spirit of Saltstack with full python throughout including Mako templating. It has a very simple set of operators mostly around idempotent file management and shell commands to do things like restart services.
This enables very fast deploys - small changes on a small number of machines in < 10 seconds.
vardalab
Yeah, but I have Claude Code or Codex do this Ansible stuff and they do just fine with all this and then there's a gazillion of examples that they can lean on and once the patterns are established, it's pretty smooth. Opus 4.5 was when the big inflection was I was heavy into automation all summer. It was Opus 4.0. It was like pulling teeth. And then when 4.5 came out, it was just beautiful.
feisty0630
"I only use tools that my LLM knows how to use" is not the flex that you think it is
rirze
Yeah, you’re gonna eat your words when you do something that’s not “install this package” and “create this user”.
For anything dynamic and sufficiently complicated, ansible is horrible. Pyinfra is much better.
ssddanbrown
I've been using PyInfra for a while, albeit just for simple automation (Updating systems, checking certain stats) and I'm a big fan. Compared to Ansible, I found the docs, syntax and usage patterns much easier to get on with. Might just be a preference thing, but I always had trouble going through the Ansible docs.
Ran into some bugs, like one machine that seems to cause errors and mess up the output on restart, although that looks like it might have been addressed in this release.
If it helps, I put together a video when initially exploring PyInfra: https://www.youtube.com/watch?v=S-_0RiFnKEs
wowi42
Glad it clicked. The Ansible vs PyInfra docs gap isn't really preference, YAML plus Jinja plus a custom DSL is just more cognitive load than plain Python with type hints. Once you can grep the source and read it like normal code, going back feels rough. On the restart bug: if it resurfaces, an issue on GitHub with the OS, connector (ssh/local/docker), and raw output would help a lot. The 3.x line cleaned up a bunch around connection handling and output buffering, so there's a decent chance it's already fixed. Thanks for the video, will watch. Hands-on intro content is exactly what the project needs more of.
catdog
Really need to try PyInfra, the concept sounds nice.
You don't have to do crazy things with Ansible for that yaml DSL becoming the opposite of helpful. Things which would be quite straightforward to express in code become quite cumbersome, hard to understand and hard to debug. Also Jinja is often a horrible choice (you don't have in Ansible). Also Ansible excessively requires it in places where you want proper types and not just a string.
Fizzadar
Pyinfra creator here - thank you so much for all the kind words, really means a lot to myself and the other maintainers, feeling the love.
V__
Has anyone used this and ansible and is able to give a short comparison with likes and dislikes?
matthiaswh
I switched from Ansible to Pyinfra for my homelab, and continue to use Ansible at work.
The biggest difference is that Pyinfra is simply Python code. It's incredibly easy to control the system in whatever manner you need to. You can probably do the same thing in Ansible, but it's never quite as obvious how to do it. This also means it's much more clear where and why things work the way they do in Pyinfra, where in Ansible I end up digging through numerous role files to try to find where some variable gets injected.
bmurphy1976
Just having "home/docker.py" instead of "collections/ansible_collections/home/dev/roles/docker/tasks/main.yml" is reason enough. Which one of the 300 main.yml files do I load when doing a quick open in any modern text editor?
SteveNuts
The worst part of Ansible is data manipulation, what would be an easy dictionary operation in Python is a huge mix of lookups in Ansible.
Incredibly frustrating that the data you want is right there but you can't easily grab it.
gchamonlive
If Jinja templating for data manipulation gets too complex or inconvenient, you can create your own module in ansible and use python code for data manipulation. But at this point you are better served with plain python which I think is where pyinfra should shine. I want to take a look though at how hard it is to implement your own module for it.
Boxxed
Pyinfra is what ansible should have been. It's straight python rather than a janky mix of yaml, templates, and bolted-on control flow primitives.
polski-g
There is this: https://github.com/seantis/suitable
But the main guy who developed it at that company left, so no idea on its longevity.
href
Oh hey that's me! Yeah I don't think you would want to use this, it was never used outside some smaller automation utils afaik.
Boxxed
Yeah I guess; I'm generally not a fan of solving a problem by adding another layer of shit on top.
wg0
I used ansible for building simple logging appliance (something like Elasticsearch + Dashboards + other tooling) and I found it very difficult to reason with specifically python code snippets within YAML.
Switched to Pyinfra and the difference is day and night. You write python code you can organise your stuff into functions, classes and whatever you like and then instantiate them as you like. Highly reusable configuration.
You have full pwoer such as you can call boto to fetch the list of servers to target, filter base on tags and what not. Only sky is the limit because it is NOT a DSL (or YAML) rather full blow real python.
TeddyDD
Adding to sibling comments: I used to use Ansible professionally and PyInfra for homelab. Ansible is ridiculously slow.
The only issue was I had to implement some facts and operations myself that probably were available in some Ansible package but to be honest it was trivial.
hylaride
At a previous job we used it to test our ansible playbooks via molecule, which were part of a CI/CD pipeline to create AWS AMIs.
It worked well and was nicer to deal with than test kitchen for testing UNIXy things (is service running and/or enabled, does file have right permissions, does file include $TEXT, etc). It was very useful for us during big linux upgrades, such as when ubuntu went from upstart to systemd. It can also be good at capturing edge cases with brittle outcomes (especially as ansible went through enormous changes after the red hat acquisition).
Dislikes? I had to fight with pyenvs a bit..
gegtik
was this before uv? i feel like my pyenv struggles basically ceased once I started using it
hylaride
I used it between 20016-2023 and since we were not a python shop, I never used any other package managers. It was never an issue with CI/CD pipeline, but iterating locally was always a fight to getting molecule to pick up the right pyenv. It got better towards the end, though.
Honestly the bigger issue was testing x86 docker images on an arm mac, as molecule didn't cleanly support cross platform images and we did pull in x86 binaries for our playbooks (by the end of my time at said company, I was also directly managed by product managers who didn't care about tech debt and I couldn't deal with the otherwise desirable idea to move our compute to ARM - a rant for another day). This may also be fixed now.
haolez
Ansible includes modules to handle cloud resources as well, such as AWS Lambda.
Propolice
Wanted to try PyInfra for a long while but this is a deal breaker: https://github.com/pyinfra-dev/pyinfra/issues/1242
DandyDev
Yeah, I understand your frustration. This is an issue with Paramiko which pyinfra relies on. We have been debating moving away from Paramiko, but it will take a while for that to materialize.
thomas-mc-work
Some time ago I've solved the same problem but using plain shell scripts instead of Python:
https://codeberg.org/common-good/welder
The desire was to have any kind of configuration management in a team with people for whom the barrier of learning the YAML based Ansible DSL was too high.
coreylane
I used ansible for years and pyinfra is very approachable since it has similar concepts, like inventories, common operations like files.put, server.shell, loving it so far, and it is quite fast
subhobroto
Congrats on shipping 3.8.0!
If you're a software engineer who wants to setup and maintain infrastructure, give PyInfra and Pulumi a go!
Huge fan of PyInfra. For my homelab, I use Pulumi with Python and PyInfra to build fully declarative intent based infrastructure. You can use actual software engineering principles like composition, inheritance, DI to setup and wire your infrastructure and services. One of the benefits of this is your infrastructure and services are now self documenting (have them write out a mermaid diagram!) and easily testable using pytest (from cheap unit tests to extensive integration tests (I use Incus)).
Instead of Pulumi, I originally used Terraform CDK with Python before CDK got IBM'd. The migration to Pulumi was refreshingly painless. My original reason for not choosing Pulumi was the crippled state of the open source, self hosted backend support a decade ago but it looks like that is now way more mature and less crippled.
PyInfra is a breath of fresh air compared to Ansible - its not just fast, it's more Pythonic, so IDE features actually work, readable, maintainable, debuggable. I call it infrastructure for software engineers.
If anyone wants to use an AI agent to try out PyInfra - One issue I've faced is that PyInfra was rearchitected in v2 (and some more in v3?) but what belongs in v1 vs v2 vs v3 isn't very clear, so an AI agent could spend a lot of time writing v1 code, having it fail and iterate to v2 and then to v3.
The official site uses the version in the URL as the namespace but it seems like the SOTA AI agents don't pay much attention to that.
Maybe writing a llms.txt for PyInfra v2, or v3 would be an extremely useful task to help with onboarding newcomers?
---
The original post by the OP https://news.ycombinator.com/user?id=wowi42:
Disclosure: PyInfra core contributor here. We just shipped 3.8.0.
PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.
The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:
from pyinfra.operations import apt, files, server
apt.packages(packages=["nginx"], update=True)
files.template(src="nginx.conf.j2", dest="/etc/nginx/nginx.conf")
server.service(service="nginx", running=True, enabled=True)PyInfra skips the eleven good minutes and goes straight to code.
Release notes in the link. Happy to answer questions.
Infrastructure as Code, not infrastructure as YAML.
DandyDev
There is a PR open for llms.txt and llms-full.txt. We'll try to merge it soon!
Disclosure: another contributor here.
subhobroto
It's amazing to see more contributors!
TBH, I was worried a few years ago that there was basically just one (original) contributor. This now gives me added trust that I'm taking the right decision to lean heavily into it.
I hope more people start using pyInfra.
Thank You for your contribution and attention!
Fizzadar
Indeed! (I am that original contributor :)), lots of work ongoing to address this, we now have a small maintainers group and are sharing out review and release loads.
DandyDev
Thanks!
There are currently 3 active maintainers incl. the creator of pyinfra. But there are many more contributors incl. repeat contributors.
jimmcslim
Can you explain how you combine PyInfra and Pulumi? It feels like these are mutually exclusive projects?
weakfish
+100 to pulumi, I love it with TypeScript
js2
I'm glad to see PyInfra is still under active development. I don't currently use PyInfra, but I previously used it for a couple years to manage a build farm of about 100 Mac Pros. Those machine had previously been partially managed by Chef to ill effect.
I found PyInfra to be a great tool for the job at hand. Even though it didn't have many of the operations I needed, I found it easy to write new operations specific to macOS management tasks.
I recently looked at it again to help build EC2 Mac AMIs in combination with Packer, but I ended up with pydoit this time instead.
rumpelstiel
Thank you for this, im using it in combo with consul to fetch the inventory and labels. So easy to setup and maintain. But im heavily using python anyway, so writing my own classes, connectors was kinda.. free.
hathym
i tried ansible before and hated it, this idea is genius.
eb0la
This reminds me of Nortel Command Console back in 2000-2005!
I worked for a telco company that had a lot of Nortel Passport devices (does anyone know what Frame Relay is?). We started changing the network from Nortel to Cisco. Cisco used telnet (later SSH), but Nortel people were extremelly reluctant to switch.
Turns out the Nortel network managment system (nortel nms) had a very interesting feature: you could open the command console to connect to one of the passport devices... or you could connect to a device group (or all the network) and run the same command in all devices.
This was great for auditing which version had every single device in the network... or for changing access-lists globally.
cpach
We had lectures on Frame Relay and stuff like that in uni, but I’ve never ever touched that stuff (:
Get the top HN stories in your inbox every day.
Disclosure: PyInfra core contributor here.
We just shipped 3.8.0.
PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.
The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:
Idempotent operations. Facts gathered from hosts, branched on with normal `if` statements. Real loops, real imports, a real debugger, real type hints. Your editor autocompletes arguments because, brace yourself, they are just function signatures.About YAML. Wonderful format. For about eleven minutes. Then someone needs an `if`, and you have `{% if %}` inside a string inside a list inside a map. Then someone types `no` as a country code for Norway and it ships to prod as `False`. Then someone indents with a tab and the parser dies without saying where. Congratulations, you reinvented a programming language. Badly. The honest move is to admit you wanted code, then write code.
PyInfra skips the eleven good minutes and goes straight to code.
Release notes in the link. Happy to answer questions.
Infrastructure as Code, not infrastructure as YAML.