BitTorrent Site Magnet Database Archives Available via Torrent and Direct DL
92 comments·May 21, 2022
DHT work is fun - you can scrape a lot of interesting content. BitTorrent and Tor both use DHTs for content discovery.
If you’re interested in building a minimum viable scrapper, this is a good reference: https://github.com/retrohacker/Taboo/blob/main/lib/index.js
The trick to content discovery on the DHT is to parrot any incoming request you receive. So if someone asks for a hash or announces a hash, you turn around and ask for that hash too. You can spin up multiple DHT nodes to increase your coverage of the table.
Some cool user experiences can be built off of this. I’ve discovered a tonne of interesting content using “BitTorrent Roulette” applications that download random hashes from the DHT - often times finding fascinating content from languages I don’t speak.
Wow, thank you for that. That's very clever and very useful.
It's a shame they use recaptcha, it's means it's impossible to script searches with something like Jackett.
well, you could run your own instance: https://github.com/btdigg-org/dhtcrawler2
Am I missing something or is the source not actually available in that repo? I only see the compiled .beam files
Ah, that is perfect. Thank you.
Badass. Thank you sir, madame, or they.
> sir, madame, or they
Neither, I think it's a cylon.
I am assembling the grand mac daddy of all magnet link dbs / seeding systems. If you want to subscribe to updates or are otherwise interested please reply here so I can gauge interest. Will post to HN when ready. This is bigger than piracy, it's about free flow of information over time for humanity.
I am up for collaborating and sharing.
First hint suggested: "interracial gangbang"
They might want to put those hints behind a NSFW warning; just my .2c.
Anyone who has spent more than 5 minutes in torrent-land has seen NSFW
that link doesn't work
Pay $40 a year for a vpn. Use a docker container that has iptables set up to only use the vpn and put a torrent client. $40 a year and nobody will bother you. The person that a vpn provider will expose to lose their company base will be a terrorist not a torrentor
I have fiber so a VPN just nukes my download speeds enough that I hated using it for torrenting.
Is this normal in your experience? I am/was a VPN noob.
Do not route everything through the VPN then, only bind your BitTorrent client to the VPN’s network interface. Also has the advantage that if the VPN connection accidentally drops, your BT client doesn’t start transmitting on your residential and expose your IP.
Get your own VPS and install OpenVPN server on it. Much faster than shared VPNs. I get gig speed on a $12/year VPS.
> Any way to know which ones might be under surveillance?
Yes, the answer is that all of them "might" be under surveillance. There's no way to guarantee that any of them aren't.
Content piracy is one of the very few valid uses of a public VPN provider service.
I receive complaints all the time by my ISP for torrenting... they do nothing except frightening older people
Not universally true though, and in some countries (like e.g. Germany) it's a really bad idea to torrent without a seedbox or vpn in another country.
What's different about Germany when it comes to piracy? And what about using a PeerGuardian blacklist to block known trackers?
If you're in the US I advise caution as ISPs are going to be increasingly aggressive about shutting off accounts. The media industry has demanded that repeated, unproven, accusations alone should be enough to force your ISP to close your account and never allowing you to open a new one. ISPs don't want to do that, but courts so far have been agreeing with the media industry on that. ISPs who don't do it, risk literal billions in fines.
IF you don't have multiple ISP options each offering reliable high speed connections in your area, you should be especially careful or you could find yourself with very limited access to the internet or even no internet access at all.
13 years ago, last I used them, Comcast would send an email with a list of the titles of the torrents it thought you downloaded.
Good enough to scare ~everyone by itself, though iirc they did threaten to halt your service, something I never decided to test out.
I tested it, it's legit, they will terminate your service.
Private torrent trackers. Check out r/trackers.
It requires time and effort though.
How does that help? YourISP is not looking at your HTTPS traffic, but your torrent traffic, right?
I don't think it's the ISP monitoring for torrent traffic, but copyright holders honeypotting torrents and keeping track of the IP of seeders, maybe leechers too I guess. If the IP holder isn't on the tracker they don't see that you're pirating their stuff.
The ISP in general isn’t going looking for your torrent traffic. The DMCA groups are, and when they find it, they contact your ISP to enforce it. If it’s a private tracker, there’s a decent chance the DMCA groups’ tracking bots aren’t able to see that traffic.
Yes as the above commenters say the copyright holders (their enforcers really) don't have access to the private trackers swarm and will be unable to report your IP address.
You can hire a 3rd party to do the torrenting and then stream from them (put.io, etc.), but there's no guarantee they can't be compelled to hand over their logs.
It's part of BEP-47 , an extension meant to make it easier to handle single file downloads and multiple torrents serving the same file. BEP-47 aware clients, which is most of them these days, I think, hide these files from the user.
> Funny, most of those torrent got a __padding_file
Those are supposed to be hidden by clients. They represent "free space" between files, allowing clients to download individual small files without having to also download the files adjacent to them.
yeah bitcomet had a similar, but incompatible scheme
seems to have a whole lot more results, for one
The one posted here is a dump from 2016 and https://torrents-csv.ml/ has imported piratebay dumps from then (probably from the same source). The difference is that https://torrents-csv.ml/ does health checks on the torrents and will remove dead ones which means that a lot of the torrents in this dump will be removed since they have probably died either before 2016 or during the last 7 years.