Transparent (de)compression
This is a large MR but brings significant improvements:
- All (de)compression is done transparently in-process
- Detected from the file extension for outputs
- Detected from the file extension and/or file magic for the standard input
- Removes a huge footgun in that you can specify a
archive.tar.gz
but without specifyingcompress
it will actually create atar
- A new
//tar/tool
provides a central tool for generaltar
manipulation - The
tar_concatenate
/tar_filter
rule implementations become simple due to//tar/tool
accepting input/outputs as arguments - Hopefully, improvements in speed as it removes the need for all the pipes
Edited by Matthew Clarkson