Tunneling Wikipedia through WhatsApp to (maybe?) get around WiFi restrictions
136 comments·May 21, 2022
That's interesting, inspired me to go down a bit of a wikipedia rabbit hole reading about internet.org, facebook zero, wikipedia zero, and zero-rating generally.
It seems mobile wikipedia is zero-rated in many countries, but by no means all, as indicated by the table on this page: https://en.wikipedia.org/wiki/Zero-rating
Though many countries don't have any info in the above table (including Laos), and I think the "Zero Wikipedia" column may be obsolete as that project was apparently shut down in 2018.
If you know anyone still involved over there you should let them know about the Kiwix project!
>Kiwix is an offline reader for online content like Wikipedia, Project Gutenberg, or TED Talks. It makes knowledge available to people with no or limited internet access. The software as well as the content is free to use for anyone.
Kiwix is nice.
Also from your comment, I feel that I definitely fail to communicate what the actual problem is.
The problem is NOT that the kids (and their teachers) cannot access certain sites. Although they might not be able to afford it, if they want.
The problem is that they DON'T KNOW that internet sites exist. For a kid who was born into the era where a magical handheld thing can show his friends' life and funny videos, a "website" means nothing. I even didn't want to write handheld television screen, because most of them haven't even seen a television before.
They don't know that wikipedia exists at all.
Another anecdote (maybe I should summarize it in a post somewhere) is that the teachers were using Oxford Press's Headway  books. Excellent series, many of us in Europe learned English from these books. It improves your vocabulary a lot, teaches you words like "coin", "train conductor", "mp3 player" and so on.
Now let's see what does NOT exist in Laos, at all: - mp3 players - TRAINS!!! - and, well, coins (Lao Kips start at 1.000 bills (10¢) IIRC and last time coins were issued was 40years ago. In the 2mo I stayed there, never saw a coin, nor the kids I was teaching).
It's really a different world, it took me some time to even start my head wrap around it
https://internet-in-a-box.org/ might come in handy.
Facebook tried the same in India but was quickly shutdown: https://www.wired.com/2016/02/facebooks-free-basics-app-is-n...
This stuff is such a gray area for me. I’m staunchly pro-net neutrality, but depriving people of the internet altogether means losing a huge asset for learning, a platform for financial success that otherwise is unattainable, and more.
But then you read about the success of Facebook in emerging markets in Africa and it becomes even murkier. Basics has caused Facebook to become utterly dominant in many African countries, and that’s precisely what net neutrality tries to prevent. That’s so much power. I don’t know what the right answer is.
Is Facebook a "huge asset for learning [and] a platform for financial success"? The utter dominance of Facebook surely prevents competitors that are better suited to these things. I am not at all sure that Facebook-only is superior to no internet at all.
Thanks I'd not heard of internet in a box, I love how it empowers people to determine their own requirements and solve their own issues rather than just be another excuse to form more dependencies on facebook.
You may try to write an email/better official mail or even visit personally to local cellulars marketing teams to include wikipedia to the subscriptions. You may propose them as PR action: future-care, education-care name it. Trust me it may definetelly works.(I did it many times("tune up" some events) but I worked inside the cellular companies) The main problem may be is to the break the "first line" of "corporative bureaucracy" defence.
It feels like this is a lost case... Since then I spent a couple months in Ecuador (and other SouthAmerican countries), and what I see, was similar:
Less fortunate people, whose (only?) entertainment is TikTok/YouTube/Facebook constantly on their phones. Well, Claro (cell provider with one of the best 3G/4G coverage on Galapagos) greets you with a Facebook(!!!) page explaining that Facebook&sister apps are brought to you for free.
There is zero incentive from any sides to fix this situation. The provider is not interested in ditching Meta, Meta is not interested in promoting anything else, and unfortunately most probably one of them (or both) already paid some gobernadores to shut up and keep the status quo of pointing people to ads, ads, and more ads.
Wikipedia contains non-disney-fied facts, like sex education and the Armenian genocide. Not sure if operators would be keen to include it.
”AT&T is subjecting our kids to blasphemy” is a real PR problem in many countries
in this instance a simple vpn or proxy should suffice. they could get by on free subscriptions for textual content
These are solutions for ME and YOU when we are there as tourists (if we really want to chop off that $10 from our travel budget). Probably not a solution for people who have never heard of websites like Wikipedia.
if you are introducing them to a new website, they will have to remember the address, or create a shortcut/homepage
vpns have "set and forget" autoconnect functionality
it's quite common practice for the more liberal East. how do you think they access the wider internet otherwise?
If they whitelisted only social media rather than blacklisting websites, I highly doubt a VPN or proxy would work unless it was provided by Google, Facebook or Tiktok.
I carry one of those tiny wireless routers in my carry on wherever I go. If I have to buy internet on the plane or if I am in a hotel that limits the number of devices, I always connect through the router and use it as an access point for all the other devices.
The other added benefit is that all my other devices already have my AP's wifi creds and will connect to it automatically.
What do you do when wifi has a captive portal though requiring a user name and password?
I use my Samsung S10 for exactly this as it has multiple radios that allows connecting to wifi and hotspotting to share that connection with other devices. Great for Chromecasting.
The first device to connect has to go through the captive portal, then the rest doesn’t.
Also works well if your 2nd device doesn’t support captive portals (Chromecast, Tesla car).
I used just clone the MAC address across my iPhone and laptop and switch between them, so I guess you could use your phone to get through the captive portal and then connect with a travel router that clones the phones MAC address.
you would need a router which supports WISP Repeater mode. some GL.inet products support this
I've been thinking about doing somethign similar and am really interested in what hardware you're using for this and what your setup looks like. Do you run a VPN service directly on your router, for example?
I use the TP-Link TL-WR902AC for this, it's cheap and has a MAC cloning feature that makes it pretty easy. Unfortunately the firmware isn't updated anymore and it's missing a VPN client, but apart from that it gets the job done.
Which one do you use?
Not GP, but I've been using the GL.iNet Beryl (GL-MT1300) and I'm really happy with it.
> Hacker News is extraordinarily resilient to lousy connections
It really is. Where I live when you run into the limit of your data package, your network is usually throttled to 100kbps. I changed my plan to just 3GB per month because I was staying at home most of the time due to the pandemic. Now I'm pretty much back to my old routines, but I didn't change my plan yet. I have a 45 minute train commute and 3GB can be used up in a few days just browsing reddit and loading news sites.
Anyway, google search, hacker news and facetime audio work as normal at 100kbps. Google maps works with a bit of patience. Virtually nothing else will load. 5 years ago most text-based things worked at this speed albeit slowly. Now everything is so bloated and so much content will not load show until fonts and things are loaded.
I was recently on a United flight and the free 1h "text only" option gave me access to the whole internet, and I could reactivate it after an hour. I think maybe they unlocked it because the flight had a delay - or it was a bug. The flight crew didn't inform us about it though. I also didn't notice any other people using it.
The connection was pretty damn good, considering I was somewhere over the Atlantic. It was shocking to me how much more enjoyable the flight was, makes me wonder how hooked I am to being connected. (I also had extra legroom and an empty seat next to me though.)
It sounds like what you're getting on a plane is actually satellite internet.
Satellite is always what you get on planes if you fly over an ocean. If you're flying over land, sometimes it's satellite and sometimes it comes from ground-based cells. Depends on the airline and the plane's equipment.
Yes, it is. One of the biggest providers is called GoGo who in turn use satellites from SES. My comment about satellite was that I've also worked in very remote places using much poorer links and HN still works, amazingly.
When selecting my personal use domain I ended spent some time finding a short domain partly because it's convenient but partly because it meant more goodput via Iodine. I ended up on "ds.gy" as ds are my initials and it was the only TLD that domain wasn't sat on by squatters wanting to charge thousands.The ratio of people wanting to sell you short domains vs actually using them in any capacity was surprising.
I did the same on trains in the 00s, but built application specific tunnels which were much faster, funnily enough among them was one that would fetch Wikipedia pages. The client would piece together the replies and render the markup to html again.
I can confirm that this still works on several US airlines especially if they have a free messaging option.
I tried Iodine around 3 years ago on a Swiss flight, it worked to read my mails over SSH using Alpine, but was so slow that basically it was unusable. Not sure what was going on, I had the impression that DNS queries were getting throttled after some threshold...
Check Mosh, Mosh works on ISDN level speeds.
I also used this a lot while travelling to access the internet through captive wifi portals. Especially in asia this worked very well, given the huge amount of telco wifi providers in cities.
lol holy shit I'm never going to have an unproductive flight again
I actually thought that's what this blog was going to be about. Some kind of http encapsulation over Whatsapp. Was disappointed that it's just regular a chat bot
I once experimented with something like that a few years back, when I was regularly using a WiFi that only allowed HTTP. It’s not hard to tunnel something like SOCKS over TLS over base64 over anything that allows sending text, including HTTP. Latency might be a lot worse than the special purpose chatbot though.
I did the same for getting round those nasty deny-by-default deep packet inspection firewalls. HTTP's request/response nature made it difficult, but it can be gotten around in-spec by pretending you are streaming back a lot of data, split over multiple packets.
> I wonder if anyone has stated a general law along the lines of "if you can send and receive a bit, you can send and receive anything."
In my country we have a telecom service provider law, which states, among other things:
ARTICLE 57. - Network neutrality. Prohibitions. Service Providers shall not:
a) Block, interfere, discriminate, hinder, degrade or restrict the use, sending, reception, offering or access to any content, application, service or protocol except by court order or explicit request of the user.
I think OP meant "law" as in "law of gravity" not as in legality.
You can simply split packets into multiple messages and tag them with a unique code and use base64 (or something more efficient), that's how you can do things like do IP over IRC which has even more restricted character counts.
The problem is always going to be bandwidth as doing any kind of communication across systems optimized for human text will throttle you: you'll trigger spam warnings, rate limits, etc - and the modern web is extremely demanding
Electrical/embedded guy here. My similar law is "if you can blink an led, you can do anything".
My take has always been: If you can't blink an LED, you can't do anything.
My first priority is to get an LED blinking, and keep it blinking. Every other feature is less important.
I blinked an LED in my cell but I'm still behind bars. Guess these dumb adages don't really hold a candle to reality.
I don't think that is necessarily a law. There would be ways to actually restrict access in better ways... you would likely be right if you amended it to the ability to send a bit to an endpoint you control.
> "if you can send and receive a bit, you can send and receive anything."
No, needs work.
Part 1 - No system can be 100%, then you hit the Two Generals' Problem..
Part 2 - Just because you can send a bit, that doesn't mean you can send 8 in a row. So you write a protocol, then they block that protocol, you adapt, they adapt etc etc
Maybe something like ~ any system where you have any control over information flow someone has written a protocol to send porn over it.
That's just jumping of Rule 34, you could change porn to something else
Vpn over websocket.. in Indonesia even worked when they "turn" off the internet for nyepi with a simple host file hack as you could browse the isp website was based on name not IP so yes the vpn was unencrypted but you couldn't see it was a vpn
Saw that page: https://en.m.wikipedia.org/wiki/Wikipedia:Database_download and where which will be suitable for ebook reader for free.
Back in 2019 when Indian government put a city of 8 million into a 9 month curfew including internet blackout, I was anxious yo get online. Then they permitted only dozen of "white listed websites", just ones that did not allow anti India propaganda. Anyways, I found amazon India worked. I went to aws, set up a vps and simply used ssh tunnel to it.
It worked, for a bit. They closed the default port and I could not spend time on a dedicated public internet terminals to " test" open ports so yeah, I have done exactly that.
Used foxyproxy btw
>Used foxyproxy btw
HAProxy is also really useful for this purpose, I dare say more-so. For my use-case it solved this problem: "Using 1 port on the remote server (port 443), how can I serve HTTPS (serve a website) and SSH or SOCKS5 (use the server as a proxy)?". HAProxy was good for the task. It could be used to tunnel SSH through HTTPS too, in the case where a corp firewall is using DPI to block standard SSH. What I'm not sure of though... can it tunnel SSH through HTTPS, and, serve a website at the same time? That's a question for the reader.
The idea of serving a website at the same time was for the purpose of providing a plausible reason for traffic exists from that server. Like, you know, if the admin's see traffic on 443 from an ip/domain with no website, that's got to be a magnitude of suspicion higher than an ip/domain with an actual website being served on it.
Sure. If you run an https proxy that allows CONNECT, that can tunnel ssh, but if you do GET without a fully qualified url, that can serve whatever according to the host header. If you just wanted to tunnel ssh over tls, it's trickier because ssh is server speaks first and http is client speaks first, so as a server, you'd have to guess if your client wants one or the other.
It doesn't work like this in the real world. There isn't some magic port that just lets you bypass the firewall.
Certain ports sometimes have laxer filtering/restrictions, for example 53, 80, and 443 - but you already know these in advance.
You aren't going to find out that port 18741 magically gets you an unrestricted internet connection.