summaryrefslogtreecommitdiff
path: root/crocoite/task.py
diff options
context:
space:
mode:
authorLars-Dominik Braun <lars@6xq.net>2018-06-25 20:30:52 +0200
committerLars-Dominik Braun <lars@6xq.net>2018-06-25 20:44:30 +0200
commit0b9824c35f2053b27c3c0d494dc1d29aac9aaa81 (patch)
treeb2dc44aee67ede80a3a5b99633f02ff89a7a1f1a /crocoite/task.py
parent785ef19736cc9a21746e00a022b76fd756c162de (diff)
downloadcrocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.tar.gz
crocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.tar.bz2
crocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.zip
warc: Add metadata to truncated records
Specifically for a) redirects (body missing) b) bodies larger than size limit and c) whenever we couldn’t fetch the response body for whatever reason. We gave it our best shot, but still failed miserably. Future generations will certainly appreciate that. Eh, maybe. Hopefully. Will they?
Diffstat (limited to 'crocoite/task.py')
0 files changed, 0 insertions, 0 deletions