Questions tagged [compression]
The compression tag has no summary.
68 questions
5
votes
5
answers
677
views
How does data store compression speed up data warehouses?
I often see the claim that various data warehouse/analytical database systems derive significant performance benefits from compressing their data stores. On the face of it, though, this seems to be ...
- 83.2k
4
votes
2
answers
621
views
How should I handle different hashes of identical files in .zip archive with different 'last changed' date?
We store zipped files in the storage of a cloud provider which contain certain fields (metadata). These files are derived from other, larger files. Every time we (re)generate these files, their 'last ...
- 269
0
votes
5
answers
236
views
Load and process (compressed) data from filesystem in the blink of an eye
We have a huge amount of queries hitting our API that request a minor or major extract of some huge files lying around on our mounted hard drives. The data needs to be extracted from the files and ...
- 493
30
votes
7
answers
17k
views
How can lossless compression ever exist?
If all data is essentially just a bit string, then all data can be represented as a number. Because a compression algorithm, c(x), must reduce or keep the same length of the input, then the compressed ...
- 475
1
vote
1
answer
2k
views
How to remove unused code from a jar file? [closed]
I have a jar file, for example foo.jar. My code contains a lot of libraries (almost 75 jar dependencies). I am not using anything like maven or gradle, I'm just using pure java with pure jar files as ...
- 23
2
votes
1
answer
1k
views
Are Flate compression in PDF and Deflate different algorithms?
I'm trying to make a program that produces pdf files. I've been studying the pdf format specification and specific pdf files whose format I'm trying to mimic. I found the line /FlateDecode in these ...
- 557
0
votes
1
answer
351
views
Short and compact barcode
I am writing a c# program where I need to print a lot of small barcodes in a 100x100 grid on a piece of paper. I then scan/photograph the paper and read the barcodes again. Each barcode only need to ...
- 127
4
votes
3
answers
261
views
Is it possible to store N bits of unique combinations, in N-1 bits? If not; why does MD5 get reprimanded for collissions?
Regarding cryptography and the issue of collisions, I posed a question as to whether it was ever possible to store every single possible combination of a bit array of a particular size, in a bit array ...
- 3,633
7
votes
2
answers
632
views
some misunderstanding in concept of Huffman algorithm
What is difference between Average length of codes and Average length of codewords in Huffman Algorithm? is both the same meaning? I get stuck in some facts:
I see a fact that marked as False:
for a ...
- 183
0
votes
1
answer
80
views
Design Question: Compression with Fast Lookup
I have a multiple files (one per CountryCode) which all get ~5000 entries added to it per day.
Each entry in the file looks like (256chars max):
{countryCode_customerId:{"ownerId": "...
- 3
7
votes
2
answers
829
views
How does conditional compilation impact product quality, security and code complexity? [closed]
Software libraries targetting resource constrained environments like embedded systems use conditional compilation to allow consumers to shave space by removing unused features from the final binaries ...
- 435
-2
votes
2
answers
551
views
Alternative to RLE for short, infrequent runs
I have 3 number arrays that I need to encode into a URL through query parameters. Example:
http://localhost:3000/?r=133223333302302040&y=10000000000000000000&b=13333332002100122331
This is a ...
- 127
2
votes
2
answers
552
views
Compressing EBCDIC file vs UTF8
Today I went across a weird case for which I have no explanation, so here I am.
I have two files with identical content, but one is encoded in UTF-8 and the other one is in IBM EBCDIC. Both of them ...
- 137
11
votes
4
answers
2k
views
How to review sql changes more effectively? [duplicate]
From my experience, sql code changes almost always tend to be NOT incremental: someone creates a new stored procedure, or modifies an entire embedded sql query for optimization purposes, or creates a ...
- 235
0
votes
2
answers
123
views
Is it possible to transfer data with a really unique seed of a psudo random number generator
I have thinking about this idea for over 5 years and i don't have the complete technical knowledge to fully grasp the idea I'm having.
The premise of the idea is to have an extremely high base number ...
- 105