summaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)AuthorFilesLines
2018-05-05Rename command line toolsLars-Dominik Braun2-124/+0
Move contrib/ scripts to .tools and add entry points to setup.py, rename crocoite-standalone to crocoite-grab.
2018-05-05contrib: Add WARC merging scriptLars-Dominik Braun1-0/+70
Very useful for distributed, recursive crawls which create one WARC per page.
2018-05-04sopel: Use recursive, distributed controllerLars-Dominik Braun1-2/+7
2018-05-04IRC plugin: Use argparseLars-Dominik Braun1-17/+33
2018-05-04Move page archiving logic to SinglePageControllerLars-Dominik Braun1-15/+12
In preparation for recursive crawls.
2018-04-20Add screenshot extraction script to contrib/Lars-Dominik Braun1-0/+54
2018-02-22irc plugin: Serialize celery operationsLars-Dominik Braun1-68/+105
This is a workaround for https://github.com/celery/celery/issues/4480
2017-12-25Increase default body sizeLars-Dominik Braun1-4/+4
2017-12-24Refactor behavior scriptsLars-Dominik Braun1-10/+7
No functional changes, just cleanup. Replaces onload and onsnapshot events. Move screen metric emulation, DOM snapshots and screenshots here as well.
2017-12-22Add simple stats-keeping SiteLoaderLars-Dominik Braun1-1/+14
2017-12-19Select default behavior scripts by site URLLars-Dominik Braun1-2/+24
2017-12-17Add distributed archivingLars-Dominik Braun1-0/+144
Using celery. Also adds a plugin for the IRC bot sopel. Code still needs some love, but it should work.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-15 23:01:03 +0000