Brian Lovin
/
Hacker News
Daily Digest email

Get the top HN stories in your inbox every day.

xoa

Are tarpits still of use these days? I sort of figured that even modern script mass attackers have gotten professionalized and sophisticated enough that they can deal with trivial timeouts and the like. I could see actual honeypots still being of use for researchers or blue teams at organizations that are real targets, and ML might even open up some interesting new ways to make those more engaging for longer. But a tarpit doesn't seem like it'd cause bother for drive-by or APT, the former are all about volume so if something takes more than a few seconds just skip it (and maybe flag it as a tarpit for punishment) and an APT will instantly recognize it too.

For individuals and smaller orgs I've sort of felt like keeping your head down, running a wg/ssh bastion with a non-standard port maybe along with single packet auth or even plain old port knocking to reduce log spam from random drive-by is more effective and attainable for places without any sort of dedicated security or even constant in-house IT staff. Running a tarpit on a VPS seems like it'd fail to bother most these days, and running it on an actual IP seems like at best it'd have no effect and at worst if it ever actually held up a scanner and the operator noticed they might decide to direct some actual attention to that IP, or at least throw a mild ddos at it for a bit. Am I wrong or out of date on that? I'm all for sticking it to bad actors and efforts to reduce the economic incentives, but in 2020 tarpits strike me as kind of obsolete with some risk to boot.

belorn

For individuals and smaller orgs the easiest and by experience the best practice is to use a certificate (or generated and never to be reused password) for ssh authentication, install server monitoring, and then simply observe if the spam from random drive-by causes enough resource drain that would validate further work. Most likely it won't.

Running a tar pit is a bit like installing a trap on a bike in order to teach bike thieves a lesson. It won't really reduce the problem, but for a lot of people the idea of vengeance gives a bit of a warm happy feeling.

toyg

> Running a tar pit is a bit like installing a trap on a bike in order to teach bike thieves a lesson.

It's more like approaching a thief and persuading him to steal some bike "just around the corner", then guiding him around endlessly. While he's following you, he's also not stealing anything from anyone, his attention (which is naturally finite) gets drained - even just a little bit - to the benefit of the community as a whole. It's not necessarily about vengeance.

kebman

Tangent story. Two friends of mine went backpacking to Amsterdam, short pants and all. When they got out of the train in the evening, a friendly guy approached them and asked if they were looking for a hotel. They said yes, so he told them to follow him. Delighted to be greeted in this way, they did.

First they went down the regular path up Damrak to get to Leidseplein, but slowly and imperceptibly the streets were getting narrower and narrower, until they finally reached a dead end. That's when the guy whirled around, flipped out a small pocket knife and wheezed to them, "Gimme all your cash! NOW!"

The guys looked at each other, and then looked at him, and then one of the guys calmly told him, "Look, we're two guys, and you're just one. Even if you get one of us, the other will beat your head in. You can't win this."

The mugger looked puzzled for a moment, but then he retorted, "Ok, give me half your money then, and nobody gets hurt!" Not wanting to be the first guy who got stabbed, they agreed that, "Fine, we'll give you half! But only if you promise to not stab us." And so the deal went down, and they had finally arrived in Amsterdam.

venatiodecorus

well what they're saying is sure you can do this but this hypothetical bike thief isn't going to follow you very far. ime this would seem to be the case. most ssh spam is very dumb, just looking for low hanging fruit w/ default credentials, and will likely move on very quickly without success.

ed25519FUUU

It sounds like you’re both trapped in this scenario.

dheera

> installing a trap on a bike in order to teach bike thieves a lesson. It won't really reduce the problem

Maybe it is reducing the problem, but not enough people are installing traps to make a noticeable difference? Or the number of new thieves is cancelling out the number of thieves being put out of business by traps? Is there data for this?

Or maybe the traps just aren't sophisticated enough?

LinuxBender

Yes, well, mostly just for entertainment. My sftp server acts in a weird way like a tarpit. Instead of tarpitting or blocking the bots, I create accounts for them with null passwords. I was hoping they might upload something interesting, but no... they just try to get a shell or try to forward ports to other sites, which I do not allow. Some of them have been connecting several times an hour for the past several years.

narrator

When I used to run my own MX on a home server, I would have bots connect and try to send mail to <random whatever>@mydomain. They never used the same email or IP and they did it for several years straight. I tried IP blocking, but the IPChains list got so long it started slowing down my server.

toxik

I had a similar issue, attempts to brute my server every minute of every day. I aggregated stats on the IP’s, basically every single one was Chinese. I blocked most of China’s IP ranges and it’s now as quite as it was in 2005.

jchook

This is why folks use techniques like greylisting and why you should almost never use a catch-all mailbox.

Spamhaus usually stops a big chunk of them too.

floatingatoll

If you rewrite those portforward destinations to your own server, do they get stuck in an endless loop?

chromaton

I've successfully used an HTTP tarpit to cut down on registration spam. The attacks were being launched from only a handful of (presumably compromised) hosts. If I blocked them, they'd switch to a different attacking host. But I discovered if I tarpitted them, they'd be slowed down to the point where they weren't a problem any more.

jchook

How did it work?

chromaton

Something like:

  if(IP==attackerIP) {
    for 10000 times
       write random byte
       sleep 10 seconds
  }

pnutjam

Don't forget fail2ban or something similar. 2 hour lockout after 4 bad tries does wonders to discourage bots.

boring_twenties

I don't like it because it opens up the possibility of someone on the same network as me locking me out of my own server.

Sure, it's unlikely, but I don't see what I'd be gaining using fail2ban in the first place. I don't leave password authentication enabled, of course.

Log spam is a bit annoying, but at the end of the day, who cares? Even with the ongoing attempts, my authlog is like 300K uncompressed today and 60-120K per day gzipped. Whatever.

If I cared about that I would prefer to just block Chinese IP ranges outright.

pnutjam

If your passing through a NAT, you can whitelist your own IP.

something like: fail2ban-client set addignoreip x.x.x.x or fail2ban-client set addignoreregex hostname.com

protomyth

Well, if you use certificates, you can immediately ban anyone trying to do password authentication.

pnutjam

True, but having supported an sftp server for other b2b clients to upload data, ssh keys are black magic to too many people. I can't count the hours I've spent trying to explain them, how to generate them and why you should never "show anyone your privates", just your publics.

dawnerd

Just an obvious protip to test that your cert works before banning password auths. I uh, found out the hard way.

jdc

Depends on your threat model and how playful you are.

spc476

About 15 years ago, I ran a tarpit at my then current job (small ISP for commercial clients and web hosting) that tarpitted all TCP traffic on unused IP blocks. It did slow down network scans and had I gotten around to it, I could have probably set up some way of automatically blocking the IP addresses hitting the tarpit (as no legitimate traffic should hit it).

pbhjpbhj

fail2ban would be the go to solution for blocking IP address used in failed port access attempts.

kachnuv_ocasek

> if something takes more than a few seconds just skip it (and maybe flag it as a tarpit for punishment)

Good. Isn't that the point? Also, I like your use of the word punishment as something negative in this context.

cornstalks

> I like your use of the word punishment as something negative in this context.

Some people decide to launch a DDOS attack or something like that in retaliation. It doesn't always happen, but there have been instances of an attacker being thwarted and then trying to punish the victim (who successfully defended themself) in some other way.

xoa

Yeah, this is what I had in mind. Right now the economics there seem in favor of attackers since botnet resources for low level DDOS are plentiful. While major players have plenty of measures for mitigation, for a residential/SOHO/SMB/smaller NPO even a very cheap DDOS or actual more focused (but still automated) hunt for vulnerabilities and unpatched targets might cause real trouble. Economics works both ways unfortunately, efforts like tarpits in principle aim to make mass scans more expensive and troublesome reducing the incentives. But attackers in turn can work to make it more expensive and troublesome to run tarpits or the like, and certainly have incentive to see them not spread. So who has the best multipliers and resources?

If "we" (both the overall world community and subsections) were able significantly reduce the resources available to attackers for DDOS making vengeance/example setting more expensive that'd help. But it seems like it's going the other way if anything :(

DarkWiiPlayer

Reminds me of the dungeon I built for web crawlers to have fun collecting email addresses at https://darkwiiplayer.com/bot-dungeon xD

rootlocus

Do you have any statistics?

What's the deepest level any bot has gotten?

DarkWiiPlayer

> Do you have any statistics?

Not really, no. If I ever rebuild my website, I will probably add some stats though :D

time0ut

I'm curious what your traffic looks like. Do you have any stats on how long clients spend traversing your dungeon? Which look like crawlers? Do they identify themselves and how so? It'd be awesome to have a stats page!

gerdesj

Typo: "yow far will YOU get?"

(sorry)

jk563

Does it only go to level 100?

nadavami

It seems to go to as many characters as you can fit in the URL. Each new character after /bot-dungeon/ is a new level.

Pretty clever!

tyingq

Nope.

5420 levels seems to be the limit:

This one, right at the limit of 5420, works:

https://darkwiiplayer.com/bot-dungeon/M1Kt80XBcvk4ofn2m2IqRV...

(YMMV, some browsers might have their own url length limitations)

This one, at 5421 levels, breaks:

https://darkwiiplayer.com/bot-dungeon/M1Kt80XBcvk4ofn2m2IqRV...

codeulike

haha I gave up at level 13

tptacek

I'm sure this was fun to put together and it seems like it's fun for people to talk about, but you can put this along with fail2ban, port knocking, and nonstandard SSH ports in the back of the attic and just (1) turn off password authentication entirely and (2) put SSH behind WireGuard. Even if you don't do step (2), step (1) eliminates the rationale for all the silly stuff people do to obfuscate their SSH installs.

teddyh

I you put it behind WireGuard, why use SSH? Why not simply use telnet instead? And use FTP for file transfers.

Drdrdrq

Could you elaborate on WireGuard part? Do you mean that users must first VPN, and only then can SSH, or something else?

tptacek

Yes. This is how SSH access to prod works in most large companies: you have to be behind the VPN to get it.

pvg

You know this but I'm just throwing it in for people who don't and aren't working on large company things:

You can give yourself a WireGuard-powered, Single Sign-on, secure overlay network between, say, your phone, your laptop, a DO droplet and an AWS instance near-instantly and for (currently) free with tailscale.

By 'near-instantly' I mean it takes almost no effort to set up. It takes me longer to get my dotfiles right on a new host.

Lex-2008

discussion of a blog post about this tool: https://news.ycombinator.com/item?id=19465967

Freaky

One I made in async Rust: https://github.com/Freaky/tarssh

I currently have 22 clients stuck it in across three machines. When I started out it was more like a thousand, so seems they've largely adapted.

nickcw

Great idea!

I'm not sure we should be writing new network connected daemons in C though.

klodolph

> I'm not sure we should be writing new network connected daemons in C though.

In general, yes. However, in this case--no, that's not helpful advice--because this program doesn't actually receive input from clients! Kind of hard to trigger exploitable behavior on a program that only sends output.

codeulike

Kind of hard to trigger exploitable behavior on a program that only sends output.

It wouldn't suprise me to find there were still possible exploits

fb03

Explaining, since you were downvoted without a proper reason:

While everything is possible, most exploits happen on buffer overflows on user-received custom data. and since this is not allocating any buffer to receive anything (besides internal connection structures that are filled by the OS), the attack/exploit surface on this one is really tiny, if existent at all.

ravi-delia

Crucially, endlessh has a smaller codebase than some shell scripts I've written. If you have ever used any program written with even a single line of Python, you have more to worry about than a 843 line program that appends a string to a socket.

kmbfjr

Your concern is well founded, but what are you going to use that doesn't end up touching libraries written in...C?

We're a long way from "Smashing the Stack", people are aware of mitigation and the care that needs to be taken, precautions have been made inside operating systems and compilers.

MCOfficer

I take the bait... rust.

young_unixer

Until there's a better alternative to C at its level of performance, people will keep using C.

geocrasher

I have to admit that I tried this and it was rather lackluster. Log output:

https://pastebin.com/4FTHRF3f

Not a lot of activity over the time I ran it, and I know that the port gets hit more than that. I had a much better time when I ran a honeypot with Kippo:

https://github.com/desaster/kippo

It was much more useful as it gave me a great list of IP's to block from all my systems ;)

mdaniel

The top of the readme for that repo advises to use the fork: https://github.com/cowrie/cowrie

k33n

The tarpit approach is a double-edged sword. Sure, you're keeping some script kiddie's machine locked up (maybe), but you're also keeping socket connections open and wasting resources on the machine they are targeting. A much more efficient approach is using fail2ban and a firewall to just drop traffic from offenders.

mtlynch

Tarpits aren't really a defense mechanism. They're meant to waste attackers' time and study their techniques, making attacks more expensive.

It's sort of like those YouTube channels where they waste phone scammers' time in an entertaining way. [0] Obviously, the easiest thing for the callee to do is hang up the phone, but their goal is to make phone scams less profitable.

[0] https://en.wikipedia.org/wiki/Jim_Browning_(YouTuber)

a1369209993

> where they waste phone scammers' time in an entertaining way.

This can also be automated, so the defender doesn't even need to waste their own time on it. Eg: https://old.reddit.com/r/itslenny/ .

spc476

Depends upon how you are doing the tarpitting. Back when I did this (15 years ago) I used a program that created a raw socket to handle all TCP traffic that just responded with 0-byte sized windows to all TCP packets (so overhead was minimal). I placed this software on a dedicated server (an old box that wasn't being used otherwise) and routed all our unused IP space to this system. It could keep thousands of connections "live" with minimal resources.

prussian

You could also do a combined approach with the tarpit + fail2ban parser that could just ban people stuck for longer than x amount of seconds.

golem14

In my experience, fail2ban does only help if there is a small number of IP addresses requests come from. I usually observed a trickle of requests from huge number of IP addresses, at most 1-2 request from each IP over the course of days.

tarpit will likely hurt yourself as the system ties up sockets for a long time and you'll run out eventually. You'd have to combine the tarpit with something to limit the number of connections you accept.

IMO, setting up ssh on another port has been useful, especially combined with port knocking. And of course turning off password auth.

dclaw

Hah, I love endlessh.... been running it for a few years now on one of my digital ocean droplets. Better to fuck with these bots. My personal record was somewhere around 23 days having one stuck.

nirui

What got my inspired here is, if a simple delay strategy can make attack harder, why not add this as a common feature in SSH?

It can be called "Initial Connection Delay": Once a new TCP connection is established, wait for an uncertain number of n seconds before read and respond to the handshake request.

password4321

One of the simplest ways to block unwanted connections is to filter on client id. I haven't seen anyone willing to change it even though I've blocked libssh, sshgo, and paprika.

Of course, this functionality is only available in non-standard SSH servers such as the one from Bitvise.

undefined

[deleted]
Daily Digest email

Get the top HN stories in your inbox every day.

Endlessh: An SSH Tarpit - Hacker News