[Slackbuilds-users] txz?

[Yth O’Gtha]

> > When it's Slackware v16, SBo could maybe upgrade to txz by default, right?
> 
> You can make this the default already on your own Slackware box
> by putting "export PKGTYPE=txz" in the environment (like, in
> /root/.bashrc or such).
> 
> > It's been Slackware standard for many years
> 
> Slackware packages are generally built once (by Pat) and downloaded
> many times (by everyone else). So it makes sense for him to save a
> slight bit of bandwidth by using XZ compression.
> 
> Built SBo packages are generally built on the system they're going to
> be installed on, or anyway aren't distributed over the Internet. The
> typical use case is to build the package (compressing it), then
> *immediately* install it (aka uncompressing it). The amount of local
> storage saved by using XZ might not outweigh the extra time spent on
> the better-but-slower XZ compression and decompression.
> 
> My gut instinct is that using .txz by default isn't worth the extra
> time spent in compression, but that's not science...
> 
> Hard numbers would be good. Maybe you (and me, and a few others) could
> actually benchmark it on real-world systems we actually use, with SBo
> packages of various sizes, and see what the results look like?
> 
> I did just time "gz -9" vs. "xz -9" for a 99MB font package, and
> the results were that xz took almost 3x as long to compress it (16.7
> sec vs. 6.5)... but the resulting compressed file was 3.2MB (the .gz
> one was 55MB). That's a single data point, can't extrapolate from it
> really.
> 
> Hopefully others weigh in with their own opinions. Anyone?

	It really depends on the data.
I've been faced with having to fetch various data, logs and binary files, from a
distant virtual machine, without much power (only 2 cores, not a lot of memory),
repeatedly, for tests.
	And bzip2 was my choice, de/compressing is almost as fast as with gzip,
while compression ratio was half-way to xz.
My requirement here was the total time spent for compression + transmission +
decompression, and it was the best using bzip2.
bzip2 is often overlooked because it is neither the faster, nor the best, but
the compromise is pretty strong.

	Today I've benchmarked makepkg on texlive-extras, my biggest package.
I `explodepkg`, then `time makepkg`. On a ramdisk, so no disk I/O perturbations.
.tbz uses lbzip2, available on -15.0 and -current, which is a threaded bzip2,
hence removing the major old flaw of bzip2.
lzma is for .tar.lzma packages, no short extension, looks like mono-threaded xz.

 # texlive-extra-2024.240409
     |  real |  user | sys | size |
 tgz |  5m16 |  5m07 | 11s | 3.0G |
 tbz |  1m15 |  6m47 | 17s | 2.8G |
 txz |  4m29 | 36m30 | 20s | 2.3G |
 tlz |  4m14 | 47m31 | 23s | 2.4G |
lzma | 29m16 | 28m29 | 24s | 2.3G |

gzip being mono-thread, bzip2 is way faster on my box, but the box is quite
strong with 8-cores, it'd still take way less time on a 2-core box.
Even txz/tlz are faster than tgz because of the 8 cores used...
	Here, the better compromise still is bzip2.
And for large-scale distribution, like official repositories, xz is the best,
achieving better compression ratio, hence less network usage, and bzip2 makes
too big packages.

	A switch to .tbz-by-default could be discussed, with strong arguments.
I'd advocate against .txz and .tlz, and I'd be ok with sticking to .tgz.

I've benchmarked enlightenment too, which is quite a big software package also,
with mixed data.

 # enlightenment-0.27.1
     |  real |  user | sys | size |
 tgz | 11s30 | 10s59 |  1s | 390M |
 tbz |  3s47 | 36s93 |  1s | 383G |
 txz |  1m50 |  3m32 |  3s | 385G |
 tlz |   37s |  3m32 |  3s | 385G |
lzma |  3m44 |  3m37 |  3s | 384G |

And lbzip2 is just the best, but I'm surprised, usually, compression ratio is
between tgz and txz (eg GraphicsMagick : txz:2.5M / tbz:3.1M / tgz:3.6M, with tbz
being the fastest).


	- Yth.