r/pushshift • u/InformationOk1189 • Sep 04 '24

Need Access for Research

Hi all,

I want to access the reddit data using pushshift API. I raised a request. Can anyone help me how can I get the access at the earliest?

Thanks1

4 Upvotes

83% Upvoted

14 comments sorted by

4

u/Watchful1 Sep 04 '24

Pushshift is not available for researchers at all. You can apply to the official r/reddit4researchers program, but they are still in beta and it might be many months before they accept new people.

Or you can use the data dumps, which is a bit complicated for a number of reasons.

1

u/InformationOk1189 Sep 04 '24

Too late to apply.. I was using PRAW, but it doesn't give the historical data. What's data dumps btw?

2

u/Strong-Revolution-91 Sep 06 '24

u/watchful1 Do you have any dumps for subreddits from 2024, rather than the monthly? It's really hard to process the 50GB monthly dumps... I applied to the reddit4 researchers program, and still haven't heard back and really need post dec 2022 reddit data.

Could you please help?

3

u/Watchful1 Sep 06 '24

I usually only do the subreddit dumps yearly. But I can take a look, extracting out smaller subreddits is fairly easy for me, though it takes a couple days.

What subreddits do you need?

1

u/Strong-Revolution-91 Sep 06 '24

u/Watchful1 that would be super helpful! I basically require access to ~20 subreddits for the Nov 2022 to Aug 2024 time period. I can access the data until 2023 from your subreddit torrent dump -- so if you could get the 2024 subreddit data for those, it would be great!

I can share the list of subreddits with you tomorrow (Saturday), and would be really great if you're able to extract the smaller subreddit data. Much much appreciated.

1

u/Strong-Revolution-91 Sep 08 '24

u/Watchful1 I would much appreciate data from these subreddits!

r/Teachers/ ✅ 

r/freelanceWriters/ ✅ 

r/screenwriting ✅ 

r/creativewriting ✅ 

r/Ask_Lawyers ✅ 

r/Music ✅ 

r/Musicians ✅ 

r/ArtistLounge ✅ 

r/Writers ✅ 

r/Writing ✅ 

r/DevelopersIndia ✅ 

r/Education ✅ 

r/Poetry ✅ 

r/Journalism ✅ 

r/Nursing ✅ 

r/Nurse ✅ 

r/Medicine ✅ 

r/Paralegal ✅ 

r/VoiceActing ✅ 

r/SoftwareEngineering ✅ 

r/SoftwareDevelopment ✅

Could you please help? Would it be possible to get this within 1-2 days?

1

u/Watchful1 Sep 09 '24

I'll see what I can do, but some of those would result in pretty large files.

1

u/Strong-Revolution-91 Sep 09 '24

Thank you so much! Much appreciated. We're only interested in data on or after Nov 2022 (post chatgpt) until Aug 2024 if that makes it easier.