Get the top HN stories in your inbox every day.
liberia
patrakov
Well, the good point of GoodbyeDPI is exactly so that it preserves your IP address. Normally, when trying to circumvent censorship, you would need a VPN server in a different country. But the downsides are that the bank will deny all transactions and call you (OK, answered, they added the VPN IP to the whitelist), that you will miss local-only content, you won't be able to register for a doctor appointment online (the city uses a geo-restricting filter out of security concerns), and you will see foreign prices (which might be 5x higher in some cases!) on sites that differentiate based on the country. Also, extra latency in games. None of that applies with GoodbyeDPI.
Edit: the above is written from the viewpoint of the past myself, before emigration to the Philippines.
5e92cb50239222b
I wonder why a VPN is the default solution (with all complications it ensues, some of which you've listed), when a simple SSH tunnel to any server in a sane location does just fine. `ssh server -D12345`, point your applications to socks5 at localhost:12345, and it's done. It's dead simple to only allow/deny those sites that you (don't) need to go through another server, and the traffic is encrypted (and optionally compressed), and looks just like another SSH connection.
I've used many other solutions (including WireGuard, etc.) on and off, but always come back to SSH.
azalemeth
It's easier to detect that someone is using a SOCKS tunnel – from memory, one way it might be exposed is if the packet TTL is incongruous [1] as I don't think SOCKS rewrites those.
At the height of the pandemic I travelled to Denmark for work (on a clinical trial) and had a UK negative covid test to report to the UK government that I hadn't got around to – whose website geo-blocks people reporting covid tests from outside a UK IP address (even if, e.g. you'd just left it and wanted to report a negative test taken the day before). A SOCKS proxy was detected and I got a "we cannot verify you are in the UK" message. A wireguard VPN worked fine.
[1] https://incolumitas.com/2021/03/13/tcp-ip-fingerprinting-for...
userbinator
point your applications to
That's the problem. Not all of them will implement tunneling their own traffic through SOCKS, and there's still other things like DNS that you might also want to go through the tunnel, but can't easily do so. A VPN sits at a lower layer, just looking like a regular network connection, so applications don't need to be aware.
vore
In China at least, the GFW can detect tunneled SSH traffic and cut it off.
tambourine_man
Thank you, I do too. I though it was only me.
SSH is very simple and there’s almost nothing a SSH tunnel can’t do.
oarsinsync
Encapsulating TCP in TCP results in exponential backoff and retransmissions in the event of loss.
OmarAssadi
For what it’s worth as well, there are other solutions than whole-network VPNs and such.
Personally, I chose to generate a domain list for V2Ray from the Russian government’s blocklist when I lived there [1].
I prefer to do that typically because it avoids the pain of the ever-growing whitelists and it allows me to keep the traffic encrypted in case someone does actually figure out that you’ve bypassed DPI. And if you use something like V2Ray or ShadowSocks, they’ll disguise the traffic much better than something like OpenVPN typically would, making it less obvious to anyone monitoring that you’re using a proxy in the first place.
There’s a load of references and pre-generated lists for different needs if anyone else is interested in doing something similar [2].
(Also, I hope this doesn’t come across as missing the point of the tool — I think it’s really useful and a good solution. I just figured I’d note some others too)
ValdikSS
GoodbyeDPI also includes Russian blacklist built from zapret-info, to apply censorship circumvention only for the websites from the list, to reduce the risk of breaking the website due to mangled traffic.
The newest issue are unlisted filtering performed on so-called TSPU DPI boxes. Two years ago we had only ISP DPI boxes, but now there's a government TSPU black box which they control themselves and block the websites/VPNs/SSH/IP ranges out-of-the-registry.
patrakov
The problem with such lists is the inherent assumption that all ISPs block the same "bad stuff". However, in reality, this is not the case, because each ISP has to implement the blocking on their own, and there are multiple DPI solutions with different sets of false positives. This "provider-specific overblocking" is especially common with IPv6.
So in addition to using such lists with one of the ISPs, I tried to detect signs of non-prevented blockage using iptables (matching on stuff like unusually-high TTL of an RST packet, or a string that occurs in the SSL certificate that they try to use for MITM - yes, they were not even consistent, or maybe there were two layers of DPI), and add the addresses learned this way to an ipset, so that next time they are routed through a VPN.
On the other ISP at a different location, just dropping all packets with ID=0 was for some time enough to avoid the censorship.
mmaunder
Why isn’t SSL working for you? You should be end-to-end encrypted and no censor or government should be able to see your https requests.
The only time this doesn’t apply is if someone controls your computer or the destination website and is able to MITM your TLS traffic. Is that what has happened?
Your HTTPS headers are not visible to anyone. So, for example, why is GoodbyDPI modifying the Host header? This is inside the end-to-end TLS encrypted connection that your ISP can’t see, and that the destination web host can’t see.
patrakov
SNI (the thing in the ClientHello packet which indicates the domain name for which to get a certificate, just in case if there is more than one SSL website on one IP) is not encrypted. DPI solutions (and even plain old Squid) can look into this without the need to break any encryption.
rogers18445
You don't control the myriad of routers between you and your server, and between the server and its destination. One reason to use commercial VPNs is to at least attempt to launder your packets.
occamrazor
Haven’t you just replaced trust in your ISP with trust in the hosting provider of your VPN?
(Depending on yourISP and provider this can be a good trade-off)
ditsuke
It's widely employed by Russian and Indian ISPs to censor access to websites. I've had a largely successful run with GoodbyeDPI 's circumvention.
izacus
More importantly, it's widely deployed by HNers in US to snoop though everything you do on your phone, tablet and computer if you dare to use their wifi.
The ops topics are full of people claiming how critically important it is for them to sniff through everything you do on "their" network for security.
You don't need to go to Russia, China or India to have your privacy violated. Just go to work.
kevin_thibedeau
Your privacy isn't violated if you avail yourself of someone else's property.
cyber_kinetist
South Korea also does this, and this tool is well known as one of the best circumvention methods (alongside with VPNs, but they cost money)
diasan
The problem / disconnect with that DPI is dead article is that corporations still want to use use, as they control their employees computers.
Hence I've recently been asked to implement a URL inspecting firewall, which implies a HTTPS intercepting MiTM proxy. They will probably require some exception list where things get passed through 'unmolested', but for a white listed sub-set of sites (say well known health, banking, etc).
The one thing which can break that is cert pinning, but then the corporations can simply mandate that apps requiring/using cert pinning not be used from the corporate machines.
There are some bumps in the techniques caused by HTTPS RRs, AltSvc and HTTP/3, but they will be worked around, at worst by forced downgrades.
The "they'll never no argument" carries no weight, as employers up front tend to inform their employees that the communications are monitored, hence giving the corporation legal cover.
That said, I do generally agree with the aim of that article, but changing the corporate mindset is a different (non technical) problem; it is all about providing the corporation with a colorable argument towards reducing their liability in certain scenarios.
DethNinja
FYI, your setup is great if you don’t trust your ISP but it doesn’t provide any privacy.
Public VPNs act as a mixer and hide your tracks. IP correlation is very easy nowadays, so persistent single connection to your own VPN won’t protect you from certain entities that correlate your traffic.
DoH is better than DNS but it doesn’t provide privacy. You should switch to DNSCrypt.
from
> I religiously route all my traffic through a VPN that I own and control.
But then you are losing the anonymizing effect of the VPN. If you are doing something illegal this is obviously bad but I guess it otherwise doesn't really matter. Companies like Mullvad have tens of thousands of users which means that the actions of one VPN IP address can not be attributed to just one person. You're just transferring the ability to surveil you from your ISP to whoever hosts your VPN server.
wruza
religiously route all my traffic through a VPN that I own and control
Can you please share an easy way to do that?
I’m usually not afraid of wading through configuration swamps, but when it comes to openvpn, I curl up in a corner crying.
Zen1th
Personally I use dsvpn (https://github.com/jedisct1/dsvpn) which is super simple to setup and use, and it's made by the guy who created libsodium.
wruza
Thanks! But from what I can see in sources (please correct me if that’s wrong), it is not a vpn, and a regular vpn client cannot connect to it. Nice idea along the lines of SOCKS proxy.
Diris
This is probably the easiest way. It uses wireguard instead of openvpn but I'm pretty sure it's better https://github.com/angristan/wireguard-install
blacksmith_tb
I have had good luck with Algo: https://github.com/trailofbits/algo
rixrax
Looking at the circumvention techniques GoodbyeDPI uses makes me want to cry. Is this really the state of DPI in 2022: changing Host to hoSt, or adding white spaces between method and URI actually works?
userbinator
I suspect it is also due to the scale at which DPI is used; every additional bit of complexity quickly adds up to increase the amount of processing power required.
progbits
Censorship companies probably can't hire good talent. I'm actually glad to see this, however it might get patched if this tool becomes too widespread.
gjulianm
It's not as much about talent but about tradeoffs. I work in traffic monitoring tools (not censorship, just observability tools for infrastructure) there's always the decision of how many edge cases you want to cover vs how fast you want your tool to go. At millions of packets per second, an extra "if" might make a big difference in the throughput you're able to monitor. So maybe it's actually reasonable to ignore the .1% that use "hoSt" instead of "Host" to avoid losing .5% of the packets.
someguydave
More importantly, using “hoSt” is a self-declaration of being an enemy of the censorship regime which is probably dangerous in Russia.
nnx
Or perhaps some good talent somehow ending up working there made sure the system is full of plausibly deniable holes.
friendlyHornet
I would like to believe that; it makes me feel warm and comfortable
userbinator
Interesting to see the two meanings of good appear in both your and your parent's comment.
Avamander
I suspect it's not even about not being able to, there's very little motivation. I had a brief contact with ZScaler who operates approximately in this area of traffic inspection, they literally have no clue and they don't care. Their service can be hot flaming trash but people will still pay them money because they check some boxes. I'm sure it applies to other companies in the same area as well.
ValdikSS
That was true 3-4 years ago but nowadays they're getting surprisingly good, and that's alarming.
Ekaros
It is harder problem that it sounds. Deep packet inspection needs to happen at some linespeed. More work you do the harder it is to process it all fast enough. You can write stuff for single packets, but when you have lot of connections happening it becomes much harder problem.
vlovich123
Isn’t this stuff typically specially built HW? I feel like an ASIC can accelerate this stuff fairly quickly although the volumes/pricing may not warrant building that. Also if you’re matching on host name there’s no reason you even need to keep up with line rate. All you need is to do is keep up with the connection establishment rate and you can always do the processing in the background and just issue a TCP reset after the fact.
vbezhenar
Can you update ASICs with new firmware?
malf
Don’t google “request smuggling vuln”; this is the state of all proxies and load balancers. Everyone tests the happy path and calls it a day.
cmgbhm
You can get really far with cheap techniques when your goal is to dissuade. The bigger concern I’d have is statistical analysis of top offender.
Every OSI layer offers more bypass techniques and is the halting problem where your goal is to get value without making everything break when a new browser comes out. You can’t cover all options as a 3rd party and get it perfect.
The higher up application layer, the easier it is to bypass. The more you try to classify without impact (dpi,ids,waf,spam,av), the easier bypasses are.
The domains that get effective like spam have quicker feedback loops. Network middle boxes have the slowest response cycle where they are explicitly called out in RFCs
<script> In a url might get blocked but <script >… bc it’s string matching and not layer aware.
kazinator
If you're spying on people who use plain HTTP, why would you expect them to be clever.
pelasaco
most the engines out there weren't made for security but performance. It's disturbing and relaxing at the same time to see how easy it is to bypass them. Something that works 100% is to multiplex a channel, changing it protocols after some packages. You do the SSL handshake, than after some amount of time, you switch it to SSH, I think something like that https://github.com/yrutschle/sslh (couldn't find the real repository that I used, but that one looks similar) could be used after the detection to bypass filters
ValdikSS
No, unfortunately almost no of these naive methods longer works. However the protocol spoofing ("fake packet" in GoodbyeDPI) with Auto-TTL is pretty effective on most ISPs of Russia, Korea, Indonesia, Turkey.
blahgeek
Haha that’s cute. Here in china the GFW now simply routes the IPs to blackhole. False positive by virtual host or CDN you ask? They don’t care :)
laughinghan
That's what domain fronting is for, and even though the GFW attempts to filter by SNI, genetic algorithms like Geneva are able to find workarounds: https://geneva.cs.umd.edu/papers/foci21.pdf
Avamander
ESNI/ESH is coming soon, I guess we'll see soon enough how that plays out.
grishka
Since the project at the OP link is mostly/initially aimed at Russia, in my own experience, it's not always just DPI — it's often DPI combined with a firewall. Roskomnadzor can order to block access to a domain, but then they can also specify an IP or a subnet instead. For example, that's what happened when they tried to block Telegram, which does not use DNS at all.
ignoramous
IP Fronting too: https://research.cloudflare.com/publications/Fayed2021/
userbinator
Looks like it uses WinDivert as the filtering driver, which says on its site that "Windows Server 2016 systems must have secure boot disabled"; not entirely surprising and a reminder of what a lot of the "security" stuff is really securing.
buwrtgu
"secure boot must be disabled" is the equivalent of "in case of permissions error, just chmod 777". secure boot literally protects the user first, and is configurable with user provided keys on the vast vast majority of open platform computers (i.e. amd64)
"prevent injection of a driver that can divert all my shit at the kernel level" is exactly what you want secure boot protecting you from. there is no limitation of user rights because the user can turn it off and/or load their own keys at will.
secure boot fear mongering is bullshit nonsense
userbinator
secure boot literally protects the user first
No it doesn't. It's merely a convenient excuse to divert attention away from the truth, which is that it prevents users from doing things like defeating DRM and modifying the system to not be so hostile to themselves in other ways.
secure boot fear mongering is bullshit nonsense
Your position is the corporate propaganda.
pedro2
Both are right -- is it so difficult to see that?
luke727
"prevent injection of a driver that can divert all my shit at the kernel level" is exactly what you want secure boot protecting you from.
The only thing Secure Boot is doing here is preventing you from loading a driver not blessed by Microsoft. They would happily bless "a driver that can divert all my shit at the kernel level", but it costs too much for the maintainer of WinDivert.
pklack
It is kind of sad that no one seems to bother enough to actually learn how to use Secure Boot to their advantage. Everyone is just disabling it the first time it gets in their way. Reminds me of how Firewalls used to be treated like 20 years ago. Yes, by default most implementations will only accept signatures from Microsoft. But the thing is: You can always enroll your own keys. My Laptop is currently booting a non-mainline Linux kernel with secure boot enabled. Just enroll your own Certificate and (automate the process to) sign binaries yourself. I really wonder why no one yet started some kind of project to provide a "community trust root" of some sorts which you could enroll on your machine.
ValdikSS
Windows Server 2016+ with Secure Boot enabled won't allow loading of any third-party non-HLK-certified drivers, whether they are signed or unsigned. That's why.
And HLK is only for device drivers, not for any regular drivers, as far as I know.
denkmoon
Sad to see this downvoted without any response, because as far as I see he's exactly right.
0xedd
No. That's just propaganda. The purpose is to further vendor lock you. MS has a minute share in the world of OS, completely outclassed by Linux. They're trying to force their way in by fooling people like you that something like Pluton is required.
The only reason they dominate the PC market share is because their spyware OS is installed by default and people don't go around switching OSes.
An anecdote about security; at my workplace, one of the top 5 security firms in the world, secure boot isn't required nor is MS. Makes you wonder.
ignoramous
Censorship bypass needn't necessarily be a security solution.
ValdikSS
Windows Server 2016+ with Secure Boot enabled won't allow loading of any third-party non-HLK-certified drivers, whether they are signed or unsigned. That's why.
Avamander
Wow, it's like the average vendor's support of SELinux or AppArmor. "Just turn it off" facepalm
mmwelt
Doesn't work in China, though:
ValdikSS
If anyone gets me a network link to China, I'm ready to test and implement new techniques.
Last time I asked why such tools are not built in China, the developer of very popular anti-censorship tool told me that there are punishment for censorship circumvention in China, and such tools which openly punch DPI could be more dangerous for the end user in the legal sense.
erebe__
Websocket tunneling is known to work against the GFW. Many people reported me that they managed to use wstunnel to bypass
https://github.com/erebe/wstunnel (linux + mac + windows)
userbinator
For that, you need something more like obfsproxy or some other steganographic tunnel, which of course also requires an endpoint on the other side much like a VPN.
jeroenhd
> Active DPI is more tricky to fool. Currently the software uses 7 methods to circumvent Active DPI:
> TCP-level fragmentation for first data packet
> TCP-level fragmentation for persistent (keep-alive) HTTP sessions
> Replacing Host header with hoSt
> Removing space between header name and value in Host header
> Adding additional space between HTTP Method (GET, POST etc) and URI
> Mixing case of Host header value
> Sending fake HTTP/HTTPS packets with low Time-To-Live value, incorrect checksum or incorrect TCP Sequence/Acknowledgement numbers to fool DPI and prevent delivering them to the destination
DPI middleboxes are truly terrible. They're incompatible with even basic TCP without any good reason. I wonder if these ISPs use the same vendors as your average "enterprise" network.
Avamander
> DPI middleboxes are truly terrible. They're incompatible with even basic TCP without any good reason.
Thanks to them things like TCP Fast Open are unfortunately still a rarity.
> I wonder if these ISPs use the same vendors as your average "enterprise" network.
Most likely, yes. Takes a bit too much effort to handle the bandwith necessary, that everyone would be able to do and sell it as a service.
navigate8310
Is there a network wide DPI circumvention tool that can be easily deployed using Docker?
erebe__
If you have a server/cheap vpc outside the blocking zone, you can use websocket tunneling.
https://github.com/erebe/wstunnel (linux + mac + windows)
NoOn3
It's only for Windows. Linux didn't supported. :(
jacooper
There is a similar project that supports Linux, but its much more focused on Russian censorship.
shmde
It IS supported on Linux, they have a tar file on the downloads page.
5e92cb50239222b
You mean the source tarball? It doesn't work on Linux, it doesn't even compile — the Makefile hardcodes CC to the MinGW compiler. Almost every .c file includes windows.h.
Looking at the project's issue list, they don't support other operating systems.
zorgmonkey
That source tarball is autogenerated by github when you make a release and it in no way implies linux compatibility, it is simply a way to download the source code without git.
denkmoon
Is this some kind of bad joke?
weinzierl
The nice thing about this is that passive mode should be undetectable.
Active mode on the other hand will probably not be useful for long because it is basically based in negligent implementation on the the DPI side and trivial to fix.
undefined
kebman
Sorry, might be a dumb question, but isn't this simply a matter of encrypting your packets or using a VPN?
ValdikSS
Several issues:
* VPNs are mostly not free, and with current situation in Russia, you can't easily pay for the Europe/US service due to absent Visa/MC service
* Popular VPN providers are getting blocked in Russia recently
* For the major websites, such as Instagram and Twitter, VPN access almost instantly triggers additional checks, cellphone number validation, account block, etc.
* VPN connection increases latency and reduces speed
While GoodbyeDPI is free and autonomous.
AND it can unblock VPNs, too! For example, the latest build can unblock blocked ProtonVPN by inserting fake TLS packet during OpenVPN TCP handshake.
negus
You certainly can pass all blocked traffic through VPN. But this is a cheaper (and because of that is more scalable) way to pass through.
kebman
Sorry, wouldn't that simply constitute security by obscurity?
psydvl
Default user just want to access blocked resources. Free VPN... Can we trust them? Paid VPN are paid.
witrak
>Free VPN... Can we trust them? Paid VPN are paid.
Paid VPNs are paid. Can we trust them more?
neurostimulant
People who live in a country with strict censorship often can't pay foreign vpn company due to the lack of payment processors support. Can't pay with visa/mastercard if you live in Russia or Iran.
undefined
Get the top HN stories in your inbox every day.
Worth reading Deep Packet Inspection is Dead: https://security.ias.edu/deep-packet-inspection-dead-and-her...
This tool is great, but I religiously route all my traffic through a VPN that I own and control. I’ve hardened the box I use to have zero logs and I don’t need to blindly trust a commercial provider whether they’ve been audited or not. There’s no way of really knowing they’re not logging in some capacity bar from being physically in their server room and inspecting their setup.
Add to the VPN a DoH resolver that I own and control too, and it makes things even better. I also block port 80 on my machine as an extra measure. No need to be using port 80 in this day and age except for captive portals which I rarely ever have to use.