Poll for the users of ZLib-Ada

Poll for the users of ZLib-Ada

[gautier_niouzes]

What is the lowest compression level that you need usually ?
Cheers
_________________________ 
Gautier's Ada programming 
http://sf.net/users/gdemont/

Re: Poll for the users of ZLib-Ada

[Björn Lundin]

On 2016-03-08 22:35, gautier_niouzes@hotmail.com wrote:
> What is the lowest compression level that you need usually ?
> Cheers
> _________________________ 
> Gautier's Ada programming 
> http://sf.net/users/gdemont/
> 

I think we use whatever the default is.
We use it to zip logfiles after they grow too large.
I guess we are happy with teh level of compression we get,
I don't think there has ever been a question raised
addressing the level.
--
Björn

Re: Poll for the users of ZLib-Ada

[gautier_niouzes]

> I think we use whatever the default is.
> We use it to zip logfiles after they grow too large.
> I guess we are happy with teh level of compression we get,
> I don't think there has ever been a question raised
> addressing the level.

Good to know. So far nobody seems to be interested in low Deflate compression levels, which is a good thing for me: less work for the upcoming Zip-Ada version, and for a future ZLib-Ada replacement derived from Zip-Ada as well :-).
I've added *.ldf, *.mdf and *sql filters to the testing script.
_________________________
Gautier's Ada programming
http://gautiersblog.blogspot.com/search/label/Ada
NB: follow the above link for a valid e-mail address 

Re: Poll for the users of ZLib-Ada

[Randy Brukardt]

<gautier_niouzes@hotmail.com> wrote in message 
news:c1750db2-cf38-404b-9722-71bd0cae115a@googlegroups.com...
>> I think we use whatever the default is.
>> We use it to zip logfiles after they grow too large.
>> I guess we are happy with teh level of compression we get,
>> I don't think there has ever been a question raised
>> addressing the level.
>
> Good to know. So far nobody seems to be interested in low Deflate 
> compression levels, which is a good thing for me: less work for the 
> upcoming Zip-Ada version, and for a future ZLib-Ada replacement derived 
> from Zip-Ada as well :-).
> I've added *.ldf, *.mdf and *sql filters to the testing script.

Wasn't the purposes of those lower levels to use less CPU time and space? 
But that hardly matters anymore, because the difference between 0.010 sec 
and 0.022 sec isn't noticable (CPUs are 1000x), and we have oceans of memory 
compared to the 1980s when ZIP was invented.

They probably need to exist because of compatibility with old ZIP files, but 
even in the 1980s I don't think they were used much (I don't recall ever 
using them on purpose, although I think PKZip would sometimes decide to use 
them itself for whatever reason).

                                    Randy.

                                     Randy.

Re: Poll for the users of ZLib-Ada

[gautier_niouzes]

On Tuesday, March 15, 2016 at 10:55:30 PM UTC+1, Randy Brukardt wrote:

> Wasn't the purposes of those lower levels to use less CPU time and space? 
> But that hardly matters anymore, because the difference between 0.010 sec 
> and 0.022 sec isn't noticable (CPUs are 1000x), and we have oceans of memory 
> compared to the 1980s when ZIP was invented.

Well, 0.022 sec is more than the double than 0.010 sec! You'll notice a kind of difference when you process large data...
Concerning memory, agreed, it is no more a concern. BTW, the zip/zlib code uses the same buffers for all compression levels.
The compression level is used to select various thresholds for the pattern matching algorithm.

> They probably need to exist because of compatibility with old ZIP files but
> even in the 1980s I don't think they were used much (I don't recall ever 
> using them on purpose, although I think PKZip would sometimes decide to use 
> them itself for whatever reason).

Old (or new) Zip files need to be *decompressed*.
You don't need a compression software to read them, and even less to choose a compression level, in the zlib sense.
What some versions of PKZip did was to choose between two formats: Shrink (LZW) and Reduce (LZ + probabilistic reduction).
For a Zip file built by that version of PKZip, you would need a decompression software that is supporting both
Shrink and Reduce formats, like Zip-Ada.

Actually there may be various strategies depending on the input data, and on various scopes.
Within the Deflate format there are 3 sub-formats for compressing chunks of data.
For instance the compression algorithm can choose to store the chunk (or the entire file)
if data is too random for being efficiently compressed.
On a larger scope, you can have a pick-and-choose strategy, as in the ReZip tool, and select
the most efficient compression (format and algorithm) for each file. Currently ReZip tries
20 variants, some with randomized parameters:

   Shrink
   Reduce_4
   Deflate_f
   Deflate_1
   Deflate_2
   Deflate_3
   External: Zip, -9
   External: 7-Zip, a -tzip -mm=deflate -mfb=258 -mpass=#RAND#(7,15) -mmc=10000
   External: 7-Zip, a -tzip -mm=deflate64 -mfb=257 -mpass=15 -mmc=10000
   External: KZIP, /rn /b0
   External: KZIP, /rn /b#RAND#(0,128)
   External: KZIP, /rn /b#RAND#(128,2048)
   External: Zip, -#RAND#(1,9) -Z bzip2
   External: 7-Zip, a -tzip -mm=LZMA -mx=9
   External: 7-Zip, a -tzip -mm=LZMA a=2 d=19 mf=bt3 fb=128 lc=0 lp=2
   External: 7-Zip, a -tzip -mm=LZMA a=2 d=#RAND#(3200,3700)k mf=bt4 fb=#RAND#(255,273) lc=2 lp0 pb0
   External: 7-Zip, a -tzip -mm=LZMA a=2 d=25 mf=bt3 fb=255 lc=7
   External: 7-Zip, a -tzip -mm=LZMA a=2 d=26 mf=bt3 fb=222 lc=8 lp0 pb1
   External: AdvZip, -a -4

_________________________
Gautier's Ada programming
http://gautiersblog.blogspot.com/search/label/Ada
NB: follow the above link for a valid e-mail address 

Re: Poll for the users of ZLib-Ada

[Randy Brukardt]

<gautier_niouzes@hotmail.com> wrote in message 
news:ac986ae0-3c3b-40be-9858-56a121b2ecba@googlegroups.com...
> On Tuesday, March 15, 2016 at 10:55:30 PM UTC+1, Randy Brukardt wrote:
>
>> Wasn't the purposes of those lower levels to use less CPU time and space?
>> But that hardly matters anymore, because the difference between 0.010 sec
>> and 0.022 sec isn't noticable (CPUs are 1000x), and we have oceans of 
>> memory
>> compared to the 1980s when ZIP was invented.
>
> Well, 0.022 sec is more than the double than 0.010 sec! You'll notice a 
> kind of difference when you process large data...

True, although I was thinking about for the human using a ZIP program and 
those being the total times. If, for compressing the your files, the total 
time is 0.022 sec vs. 0.010 sec, the human is not going to be able to notice 
the difference.

After all, our first PC compiler took 7 days to process the ACVC. Today, the 
ACATS is much larger than those ACVCs, and the total processing time is 
about 90 minutes (and more than an hour of that is from real-time tests with 
lengthy delay statements).

...
>> They probably need to exist because of compatibility with old ZIP files 
>> but
>> even in the 1980s I don't think they were used much (I don't recall ever
>> using them on purpose, although I think PKZip would sometimes decide to 
>> use
>> them itself for whatever reason).
>
> Old (or new) Zip files need to be *decompressed*.
> You don't need a compression software to read them, and even less to 
> choose a compression level, in the zlib sense.

I was thinking that the library did both, as it seems weird to have a 
library that only works in one direction. (But I haven't looked at your Zip 
stuff in a long time, so I forgotten which part does what.)

My somewhat uninformed take:
(1) For compression, don't stress. It's unlikely to make enough difference. 
And if you *really* care about the best possible compression, you'll have to 
run trials.
(2) For uncompression, one wants to support everything that you might 
encounter. Which would naturally tend to mean that you'd support it for 
compression as well (at least so one can test that it works). But probably 
not be default or even by trial, only if forced somehow.

> What some versions of PKZip did was ...

That's probably what I remembered. It's been a long time since I really 
cared (the packing of a bunch of files in one, not the compression, is what 
I tend to use ZIP for these days).

                             Randy.