Compression (I) [eng]

So, the Warp3D libraries are quite large in size, we need to find a trick to fit in our future Kickstart or firmware... Of course, one of the solutions is to compress them.

I just realized some tests with the xPK libraries. I use the excellent xpackbest (on Aminet) which allows for a given file to quickly find its most powerful compression library :

Here's the rank of the best compression rates :

• W3D_Permedia2.library v4.3 beta 5 (561 364 bytes)

1. shr3.100 = 54 748 bytes (gain 91%)
2. bzp2.100 = 56 536 bytes (gain 90%)
3. lzcb.100 = 65 868 bytes (gain 89%)
4. shri.100 = 68 244 bytes (gain 88%)
5. tdcs.100 = 84 088 bytes (gain 86%)
6. gzip.100 = 87 788 bytes (gain 85%)
7. crm2.100 = 98 688 bytes (gain 83%)
8. mash.100 = 104 212 bytes (gain 82%)
9. lzw2.100 = 106 484 bytes (gain 82%)
10. ppmq.100 = 107 324 bytes (gain 81%)
11. lzw5.100 = 109 224 bytes (gain 81%)
12. lzw4.100 = 110 576 bytes (gain 81%)
13. lin4.100 = 110 012 bytes (gain 81%)
14. rake.100 = 111 192 bytes (gain 81%)
15. frht.100 = 111 200 bytes (gain 81%)
16. lzw3.100 = 111 200 bytes (gain 81%)
17. crms.100 = 113 036 bytes (gain 80%)
18. lin3.100 = 115 144 bytes (gain 80%)
19. impl.100 = 117 048 bytes (gain 80%)
20. sasc.100 = 123 908 bytes (gain 78%)
21. shid.100 = 124 748 bytes (gain 78%)
22. shsc.100 = 126 360 bytes (gain 78%)
23. lin2.100 = 127 300 bytes (gain78%)
24. lin1.100 = 136 368 bytes (gain 76%)
25. lzbs.100 = 143 896 bytes (gain 75%)
26. nuke.100 = 143 916 bytes (gain 75%)
27. duke.100 = 143 984 bytes (gain 75%)
28. sqsh.100 = 153 652 bytes (gain 73%)
29. pwpk.100 = 158 684 bytes (gain 72%)
30. sdhc.100 = 175 484 bytes (gain 69%)
31. ilzr.100 = 216 664 bytes (gain 62%)
32. lhlb.100 = 260 756 bytes (gain 54%)
33. fast.100 = 273 344 bytes (gain 52%)
34. dmcb.100 = 279 604 bytes (gain 51%)
35. slz3.100 = 280 212 bytes (gain 51%)
36. acca.100 = 299 920 bytes (gain 47%)
37. rdcn.100 = 301 648 bytes (gain 47%)
38. blzw.100 = 316 924 bytes (gain 44%)
39. zeno.100 = 386 256 bytes (gain 32%)
40. hfmn.100 = 530 028 bytes (gain 6%)
41. huff.100 = 532 916 bytes (gain 6%)
42. smpl.100 = 550 204 bytes (gain 2%)
43. cbr0.100 = 561 520 bytes (gain 0%)
44. frle.100 = 565 852 bytes (gain 0%)
45. fbr2.100 = 565 984 bytes (gain 0%)
46. rlen.100 = 566 108 bytes (gain 0%)

• W3D_AvengerLEMU.library v4.5 beta 2 (189 664 bytes)

1. gzip.100 = 25 916 bytes (gain 87%)
2. shr3.100 = 26 024 bytes (gain 87%)
3. shri.100 = 26 048 bytes (gain 87%)
4. lzcb.100 = 26 508 bytes (gain 87%)
5. mash.100 = 29 488 bytes (gain 85%)
6. crm2.100 = 29 648 bytes (gain 85%)
7. tdcs.100 = 29 792 bytes (gain 85%)
8. ppmq.100 = 30 040 bytes (gain 85%)
9. sasc.100 = 31 444 bytes (gain 84%)
10. rake.100 = 31 756 bytes (gain 84%)
11. frht.100 = 31 828 bytes (gain 84%)
12. impl.100 = 32 716 bytes (gain 83%)
13. shid.100 = 32 752 bytes (gain 83%)
14. bzp2.100 = 33 000 bytes (gain 83%)
15. shsc.100 = 33 436 bytes (gain 83%)
16. crms.100 = 35 244 bytes (gain 82%)
17. lzw2.100 = 35 368 bytes (gain 82%)
18. lin4.100 = 35 932 bytes (gain 82%)
19. lzw5.100 = 36 280 bytes (gain 81%)
20. lzw4.100 = 36 540 bytes (gain 81%)
21. lin3.100 = 37 128 bytes (gain  81%)
22. nuke.100 = 37 736 bytes (gain 81%)
23. lzw3.100 = 37 844 bytes (gain 81%)
24. sqsh.100 = 41 572 bytes (gain 79%)
25. lzbs.100 = 41 784 bytes (gain 78%)
26. lin2.100 = 42 256 bytes (gain 78%)
27. lhlb.100 = 43 044 bytes (gain 78%)
28. lin1.100 = 43 184 bytes (gain 78%)
29. duke.100 = 43 712 bytes (gain 77%)
30. pwpk.100 = 44 684 bytes (gain 77%)
31. sdhc.100 = 49 596 bytes (gain 74%)
32. fast.100 = 52 816 bytes (gain 73%)
33. slz3.100 = 55 320 bytes (gain 71%)
34. acca.100 = 55 540 bytes (gain 71%)
35. rdcn.100 = 56 052 bytes (gain 71%)
36. ilzr.100 = 56 644 bytes (gain 71%)
37. zeno.100 = 81 032 bytes (gain 58%)
38. dmcb.100 = 83 112 bytes (gain 57%)
39. blzw.100 = 88 028 bytes (gain 54%)
40. hfmn.100 = 141 416 bytes (gain 26%)
41. huff.100 = 142 516 bytes (gain 25%)
42. smpl.100 = 176 108 bytes (gain 8%)
43. frle.100 = 188 248 bytes (gain 1%)
44. fbr2.100 = 188 276 bytes (gain 1%)
45. rlen.100 = 188 324 bytes (gain 1%)
46. cbr0.100 = 188 368 bytes (gain 1%)

Excellent news : the libraries compress very well, the gain in size is huge, which suits us well because the space in our Kickstart is still limited !

Finally we need to choose the decompressing routine : ideally it is needed to be quite small... For example, the 'mash' is about 490 bytes while the 'shri' is still 3.2 Kb !

(translated by Squaley)