As a little weekend project, I put together a Python utility for backing up writing.com interactives into locally browsable html pages, much in the same manner that rarrawer archives them.
I too am saddened by the occasional deletion of a Writing.com story, and also the "interactive stories are temporarily unavailable..." messages, so I figured why not make something to keep copies of them and access them off-line... FOREVER!!!.
You can find the project here: https://github.com/SolRacht/writing.com-archival. For those not github-inclined, here's a link to the sources: https://github.com/SolRacht/writing.com-archival/archive/master.zip. Using the utility will require access to a python3 interpreter. There are instructions in the readme.
Anyway, I know some of you already have your own solutions to backing up these stories. Just thought I'd share mine. Its "killer feature", I think, is the ability to save every interactive from search results. You can also easily update an existing archive with new chapters.
(btw, one great benefit of having a big archive of stories is being able to grep through them!)
Utility for backing up writing.com interactives
6 posts
• Page 1 of 1
Re: Utility for backing up writing.com interactives
This looks a whole lot better than the code i've written to to the job, which just automates running wpull to output WARC files for each interactive.
I just finished grabbing everything from the interactive story listing and it's over 30GB due to all the html bloat.
Habeetwenty, I'd like to talk code and archival with you sometime, maybe collaborate and share resources.
I just finished grabbing everything from the interactive story listing and it's over 30GB due to all the html bloat.
Habeetwenty, I'd like to talk code and archival with you sometime, maybe collaborate and share resources.
- rarrawer
- Somewhat familiar
- Posts: 120
- Joined: Wed Aug 09, 2006 11:00 pm
Re: Utility for backing up writing.com interactives
Sounds great! I'll PM you.
-
Habeetwenty - New to the forum
- Posts: 6
- Joined: Sat Jul 02, 2011 10:27 pm
Re: Utility for backing up writing.com interactives
Cool beans, fam. Thanks!
-
Spunkomatic66 - Participator
- Posts: 275
- Joined: Sun Jul 24, 2016 1:10 pm
Re: Utility for backing up writing.com interactives
Does anyone have alot of the deleted interactives saved? There are alot that I really enjoyed that are no longer accessible and in this case waybackmachine is utterly useless
-
asdfnow4never - Been posting for a bit
- Posts: 26
- Joined: Tue Jan 13, 2009 3:36 pm
Re: Utility for backing up writing.com interactives
I have lots saved but don't know how to easily share what I've got.
I'm kind of working on it, but IRL stuff needs doing.
Anyone who cares enough to write code or otherwise help out with technical stuff is welcome to offer suggestions.
I currently plan to process and upload the interactives I've got.
After this upload it will require retooling to do again though, since I've changed how I save things.
I'd share the raw files too, but moving a hundred GB of HTML files isn't a trivial task.
UPDATE:
Due to some changes to how writing.com does chapter pages, I'll need to do significant rewrites instead of just writing more regular expressions in order to parse chapters from 2016.
I plan to code this stuff at some undetermined point in the future due, but don't hold your breath.
I'm kind of working on it, but IRL stuff needs doing.
Anyone who cares enough to write code or otherwise help out with technical stuff is welcome to offer suggestions.
I currently plan to process and upload the interactives I've got.
After this upload it will require retooling to do again though, since I've changed how I save things.
I'd share the raw files too, but moving a hundred GB of HTML files isn't a trivial task.
UPDATE:
Due to some changes to how writing.com does chapter pages, I'll need to do significant rewrites instead of just writing more regular expressions in order to parse chapters from 2016.
I plan to code this stuff at some undetermined point in the future due, but don't hold your breath.
- rarrawer
- Somewhat familiar
- Posts: 120
- Joined: Wed Aug 09, 2006 11:00 pm
6 posts
• Page 1 of 1