summaryrefslogtreecommitdiff
path: root/crocoite/tools.py
AgeCommit message (Collapse)AuthorFilesLines
2018-12-08tools: Add version info to merged WARCsLars-Dominik Braun1-1/+23
In preparation for #9. I was hoping to reuse one of schema.org’s microdata schema’s, but neither Action (archival action) nor SoftwareApplication (version information) seem to be suitable.
2018-11-19Coding styleLars-Dominik Braun1-1/+1
Fix a few random issues pointed out by pylint, mainly unused imports.
2018-11-17tools: Add original HTTP header to revisit recordLars-Dominik Braun1-1/+4
The payloads may be the same, but the headers are usually not.
2018-11-10tools: Fix WARC mergingLars-Dominik Braun1-18/+17
WARC-Target-URI was taken from the previous record, even if the URI was different. This essentially removes the revisited URL from the archive. Also add a few tests. And boy, warcio is a mess.
2018-06-25warc: Save DOM-/image screenshot as WARC conversionLars-Dominik Braun1-13/+16
Judging from the docs this is the proper way to store these resources. Enable both for the IRC bot by default, since they won’t interfere with IA’s wayback machine.
2018-05-05Rename command line toolsLars-Dominik Braun1-0/+97
Move contrib/ scripts to .tools and add entry points to setup.py, rename crocoite-standalone to crocoite-grab.