Utility for backing up writing.com interactives

This forum revolves around interactive vore stories. You are free to promote interactive stories on this site and other sites and discuss them here.

Utility for backing up writing.com interactives

Postby Habeetwenty » Mon Sep 12, 2016 9:45 pm

As a little weekend project, I put together a Python utility for backing up writing.com interactives into locally browsable html pages, much in the same manner that rarrawer archives them.

I too am saddened by the occasional deletion of a Writing.com story, and also the "interactive stories are temporarily unavailable..." messages, so I figured why not make something to keep copies of them and access them off-line... FOREVER!!!.

You can find the project here: https://github.com/SolRacht/writing.com-archival. For those not github-inclined, here's a link to the sources: https://github.com/SolRacht/writing.com-archival/archive/master.zip. Using the utility will require access to a python3 interpreter. There are instructions in the readme.

Anyway, I know some of you already have your own solutions to backing up these stories. Just thought I'd share mine. Its "killer feature", I think, is the ability to save every interactive from search results. You can also easily update an existing archive with new chapters.

(btw, one great benefit of having a big archive of stories is being able to grep through them!)
User avatar
Habeetwenty
New to the forum
 
Posts: 6
Joined: Sat Jul 02, 2011 10:27 pm

Re: Utility for backing up writing.com interactives

Postby rarrawer » Tue Sep 13, 2016 4:00 pm

This looks a whole lot better than the code i've written to to the job, which just automates running wpull to output WARC files for each interactive.
I just finished grabbing everything from the interactive story listing and it's over 30GB due to all the html bloat.
Habeetwenty, I'd like to talk code and archival with you sometime, maybe collaborate and share resources.
rarrawer
Somewhat familiar
 
Posts: 120
Joined: Wed Aug 09, 2006 11:00 pm

Re: Utility for backing up writing.com interactives

Postby Habeetwenty » Tue Sep 13, 2016 5:24 pm

Sounds great! I'll PM you.
User avatar
Habeetwenty
New to the forum
 
Posts: 6
Joined: Sat Jul 02, 2011 10:27 pm

Re: Utility for backing up writing.com interactives

Postby Spunkomatic66 » Tue Sep 13, 2016 10:40 pm

Cool beans, fam. Thanks!
User avatar
Spunkomatic66
Participator
 
Posts: 275
Joined: Sun Jul 24, 2016 1:10 pm

Re: Utility for backing up writing.com interactives

Postby asdfnow4never » Sun Sep 18, 2016 7:20 pm

Does anyone have alot of the deleted interactives saved? There are alot that I really enjoyed that are no longer accessible and in this case waybackmachine is utterly useless :(
User avatar
asdfnow4never
Been posting for a bit
 
Posts: 26
Joined: Tue Jan 13, 2009 3:36 pm

Re: Utility for backing up writing.com interactives

Postby rarrawer » Thu Sep 22, 2016 1:09 am

I have lots saved but don't know how to easily share what I've got.
I'm kind of working on it, but IRL stuff needs doing.
Anyone who cares enough to write code or otherwise help out with technical stuff is welcome to offer suggestions.

I currently plan to process and upload the interactives I've got.
After this upload it will require retooling to do again though, since I've changed how I save things.
I'd share the raw files too, but moving a hundred GB of HTML files isn't a trivial task.

UPDATE:
Due to some changes to how writing.com does chapter pages, I'll need to do significant rewrites instead of just writing more regular expressions in order to parse chapters from 2016.
I plan to code this stuff at some undetermined point in the future due, but don't hold your breath.
rarrawer
Somewhat familiar
 
Posts: 120
Joined: Wed Aug 09, 2006 11:00 pm


Return to Interactive story

Who is online

Users browsing this forum: No registered users