diff options
| author | Lars-Dominik Braun <lars@6xq.net> | 2018-06-25 20:30:52 +0200 |
|---|---|---|
| committer | Lars-Dominik Braun <lars@6xq.net> | 2018-06-25 20:44:30 +0200 |
| commit | 0b9824c35f2053b27c3c0d494dc1d29aac9aaa81 (patch) | |
| tree | b2dc44aee67ede80a3a5b99633f02ff89a7a1f1a /crocoite/task.py | |
| parent | 785ef19736cc9a21746e00a022b76fd756c162de (diff) | |
| download | crocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.tar.gz crocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.tar.bz2 crocoite-0b9824c35f2053b27c3c0d494dc1d29aac9aaa81.zip | |
warc: Add metadata to truncated records
Specifically for a) redirects (body missing) b) bodies larger than size
limit and c) whenever we couldn’t fetch the response body for whatever
reason.
We gave it our best shot, but still failed miserably. Future generations
will certainly appreciate that. Eh, maybe. Hopefully. Will they?
Diffstat (limited to 'crocoite/task.py')
0 files changed, 0 insertions, 0 deletions