Mastouille

0 message0 participant0 message aujourd’hui

Farooq | فاروقSo <a href="https://polymaths.social/@rl_dane" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@rl_dane</a> introduced <a href="https://cr8r.gg/tags/bzip3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bzip3</a> to me to use instead of <a href="https://cr8r.gg/tags/bzip2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bzip2</a>. Let's turn some <code>bz2</code> files into <code>bz3</code> to see the difference.First example: 90k opus fileshey snips wake word dataset. It has ~90k opus files and a tar file of <code>3.1GB</code>. <code>bzip2</code> produces the same <code>3.1GB</code> which is as expected. bzip3 created <code>3.0GB</code> but used tons of computation power. Not worth the <code>100MB</code>Second example: Windows 7 virtual box VM image<code>Windows7.vdi</code> it's Windows 7 VM image for the "special" days. I think I have to get rid of it. But while it is still there, let's see how each will perform. It is <code>16GB</code> uncompressed. <code>bzip2 -9</code> is <code>7.0GB</code>. <code>bzip3</code> is <code>6.3GB</code> but at the expense of like 3x CPU time. Deleting all of them anyway. Down with Windows.Third example: Pure XML text filePure XML file. It's Persian and English characters. Uncompressed is <code>1.7GB</code>. <code>bzip2 -9</code> is <code>276M</code> while <code>bzip3</code> is <code>260MB</code>Final example: Creating a simple bombSo I did this:<pre><code>dd if=/dev/zero of=./justzero bs=2G count=6 </code></pre>So now I have a <code>16GB</code> with only zero bytes. <code>bzip2 -9</code> is <code>672KB</code>. <code>bzip3</code> is <code>46KB</code>. ConclusionThank you <a href="https://polymaths.social/@rl_dane" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@rl_dane</a> Real nice thing!<a href="https://cr8r.gg/tags/compression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#compression</a> <a href="https://cr8r.gg/tags/gzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#gzip</a> <a href="https://cr8r.gg/tags/zip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#zip</a> <a href="https://cr8r.gg/tags/filecompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#filecompression</a> <a href="https://cr8r.gg/tags/textcompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#textcompression</a> <a href="https://cr8r.gg/tags/datacompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#datacompression</a> <a href="https://cr8r.gg/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#linux</a> <a href="https://cr8r.gg/tags/unix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#unix</a> <a href="https://cr8r.gg/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a>

Farooq | فاروقSo I was short on storage on my archive drive. I saw librewolf source code. It was <code>tar.gz</code> and ~800MB. I uncompressed it then recompressed it with <code>bzip2 -9</code> and now it's ~600MB. Generally <a href="https://cr8r.gg/tags/bzip2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bzip2</a> has better compression for such these data than <a href="https://cr8r.gg/tags/gzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#gzip</a>.Edit: But don't do <code>bzip2 -9</code> all and everywhere. Sometimes <code>-4</code> is the same as <code>-9</code> however the latter being tons slower. Also there is <code>pbzip2</code> for using all your CPU cores. <a href="https://cr8r.gg/tags/compression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#compression</a> <a href="https://cr8r.gg/tags/compressionalgorithm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#compressionalgorithm</a> <a href="https://cr8r.gg/tags/textcompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#textcompression</a> <a href="https://cr8r.gg/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#opensource</a> <a href="https://cr8r.gg/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a> <a href="https://cr8r.gg/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#linux</a> <a href="https://cr8r.gg/tags/unix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#unix</a> <a href="https://cr8r.gg/tags/bsd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bsd</a> <a href="https://cr8r.gg/tags/freebsd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#freebsd</a>

IT NewsAI Can Now Compress Text - There are many claims in the air about the capabilities of AI systems, as the tech... - <a href="https://hackaday.com/2024/04/28/ai-can-now-compress-text/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://hackaday.com/2024/04/28/ai-can-now-compress-text/</a> <a href="https://schleuss.online/tags/artificialintelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#artificialintelligence</a> <a href="https://schleuss.online/tags/textcompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#textcompression</a> <a href="https://schleuss.online/tags/llama" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llama</a>

IT NewsText Compression Gets Weirdly Efficient With LLMs - It used to be that memory and storage space were so precious and so limited of a r... - <a href="https://hackaday.com/2023/08/27/text-compression-gets-weirdly-efficient-with-llms/" rel="nofollow noopener noreferrer" target="_blank">https://hackaday.com/2023/08/27/text-compression-gets-weirdly-efficient-with-llms/</a> <a href="https://schleuss.online/tags/artificialintelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#artificialintelligence</a> <a href="https://schleuss.online/tags/losslesscompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#losslesscompression</a> <a href="https://schleuss.online/tags/lossycompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#lossycompression</a> <a href="https://schleuss.online/tags/textcompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#textcompression</a> <a href="https://schleuss.online/tags/softwarehacks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#softwarehacks</a> <a href="https://schleuss.online/tags/neuralnetwork" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#neuralnetwork</a> <a href="https://schleuss.online/tags/compression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#compression</a> <a href="https://schleuss.online/tags/winzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#winzip</a> <a href="https://schleuss.online/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llm</a>

Recherches récentes

Options de recherche

Administré par :

Statistiques du serveur :

#textcompression