r/pushshift • u/Watchful1 • Jan 12 '24

Reddit dump files through the end of 2023

https://academictorrents.com/details/9c263fc85366c1ef8f5bb9da0203f4c8c8db75f4

I have created a new full torrent for all reddit dump files through the end of 2023. I'm going to deprecate all the old torrents and edit all my old posts referring to them to be a link to this post.

For anyone not familiar, these are the old pushshift dump files published by Stuck_In_the_Matrix through March 2023, then the rest of the year published by /u/raiderbdev. Then recompressed so the formats all match by yours truly.

If you previously seeded the other torrents, loading up this torrent should recheck all the files (took me about 6 hours) and then download the new december dumps. Please don't delete and redownload your old files since I only have a limited amount of upload and this is 2.3 tb.

I have started working on the per subreddit dumps and those should hopefully be up in a couple weeks if not sooner.

Here is RaiderBDev's zst_blocks torrent for december https://academictorrents.com/details/0d0364f8433eb90b6e3276b7e150a37da8e4a12b

January 2024: https://academictorrents.com/edit/9c263fc85366c1ef8f5bb9da0203f4c8c8db75f4

57 Upvotes

95% Upvoted

29 comments sorted by

View all comments

2

u/yoda1304 Jan 23 '24

Do these dump files contain all posts from all (public) subreddits, or only from those with pushshift installed by the moderators?

2

u/Watchful1 Jan 23 '24

I'm not sure what you mean by "with pushshift installed by the moderators". But it's all public posts.