crocoite.git - Web archiving using Google Chrome

Age	Commit message (Collapse)	Author	Files	Lines
2017-12-25	Increase default body size	Lars-Dominik Braun	1	-2/+4

2017-12-24	Refactor behavior scripts	Lars-Dominik Braun	1	-2/+3
	No functional changes, just cleanup. Replaces onload and onsnapshot events. Move screen metric emulation, DOM snapshots and screenshots here as well.
2017-12-22	Add simple stats-keeping SiteLoader	Lars-Dominik Braun	1	-4/+6

2017-12-22	Don’t write WARC record if body cannot be retrieved	Lars-Dominik Braun	1	-19/+48
	+refactoring.
2017-12-20	Fix HTTP headers using the same key more than once	Lars-Dominik Braun	1	-2/+15
	This is an undocumented DevTools feature.
2017-12-19	Serialize WARC writing	Lars-Dominik Braun	1	-0/+35
	Logger and SiteWriter both access .write_record() concurrently, which can corrupt WARC files. Move the writer to its own thread and decouple it with a queue. Since we’re probably I/O-bound this may speed up writeback as well.
2017-12-17	Don’t fetch redirected request body	Lars-Dominik Braun	1	-8/+12
	We can’t do that safely due to a race-condition.
2017-11-29	Use Chrome’s timestamps as WARC-Date	Lars-Dominik Braun	1	-0/+6

2017-11-29	Refactoring	Lars-Dominik Braun	1	-0/+174
	Reusable browser communication and WARC writing.