I think I may be onto something here… I’ve developed a prototype lossless compression algorithm that can compress any audio/video file to 50-60% of it’s original size (in fact it doesn’t have to even be video or audio, can be any compressed format).
EDIT : Never mind, it didn’t work out 
How have I never noticed you’re in Auckland!?
Because I arrived last Monday 
nice! welcome… back?
Ehh… never been here before… immigrating
Nice! popular choice. I wont derail this thread any further!
algorithm sounds interesting. Im sure there would be a use.
Well even if a simple windows tool for compressing video and MP3 files, it could be useful.
Since regular Zip, RAR etc… don’t compress them much at all.
What kind of compression is this? Can you read the files as a stream, or do you have to completely decompress them before you can use them?
How does it go on wave files?
It’s my own custom compression algorithm that uses a bit of out of the box thinking, it’s not really any ‘established’ algorithm I can point you to. The files will be in a custom format, so they will need to be completely decompressed, in fact they could be read as a stream, but I have yet to implement that to see if it’s feasible.
Compression is the same, 50-60% on any file, no matter the format. So a WAV will be 50-60% compressed.
Though I have a good level of trust and respect for you Meltdown, the first thing that came to mind when I read your original post was this epic thread on gamedev
Its a bit long, but worth reading every page, not just for the LOLS, but for the useful information contained within.
So I’m afraid i’m rather sceptical of a lossless routine able to achieve 50-60% compression on any file, particularly already compressed files such as audio/video which are by that point inherently files containing random bit patterns. To be able to beat existing compression algorithms so comprehensibly would be a major feat. So unless you are just kidding around and i’ve missed the ‘smiley’, or you happen to be an expert in information theory, please give it a good read.
The thing is… I don’t use any well-known or regular compression ‘algorithm’ inherently, the way I’ve done it is purely out of the box thinking that uses several layers of manipulating the data so that it ends up with a smaller file size. The funny thing about it, is it’s quite a simple process really, it’s nothing majorly technical, the process could be explained in under 3 minutes. I will have a working alpha/beta in the coming weeks or so and you’ll have the opportunity to test it then.
I don’t expect you to believe the ‘impossible’ or something that is ‘ground breaking’ has been done, but the concept does work. Believe it or not.
LOL - from memory your first paragraph is almost exactly the same thing the guy in the thread said. To which someone commented well sure if you manipulate the file so all the 1’s are at the start and all the 0’s are at the end then it will compress amazingly well, but obviously its impossible to decompress back to the original file. 
I’m also not saying that I think you are wrong or even deluding yourself, or that you are unable to come up with something ground-breaking (like I said from memory your nickname is not one I associate with crazy idea’s, spamming, trolling etc) but i’m afraid there is a very high chance that you’ve made a mistake, just like the guy did (several times) in that thread.
I could believe you in terms of uncompressed data, though here you have issues of entropy to deal with, but in terms of already compressed data, which by its very nature produces random bit patterns it starts to become very unlikely. Of course lossy compression can achieve this level or greater, but then its not applicable to ‘any’ file. Can’t have your word document saved as lossy
However i’m no expert on information theory or advance compression, but the one thing that tells me it is highly unlikely you’ve discovered a new method of compression is that its a highly competitive field with unimaginable sums of money to be made for someone who does come up with a means of compression video/audio by an additional 50% (or smaller value), especially one that supports streaming as it would be snapped up in an instant.
In other words, if this works I would have expected it to already have been found and developed.
So again, please take the time to read through that thread. You may discover an issue mentioned there that applies to your ‘out of the box’ thinking.
That guy was claiming a 5.5Mb file to 16k, which definately seems ridiculously impossible.
I’m claiming an 18mb file compacting to 11mb, far more reasonable.
The concept does work, it now needs a pretty UI, and some performance optimizations to make compressing/uncompressing quicker.
Very true, but like I said you are battling an established branch of applied mathematics and computer science - Information Theory.
It doesn’t matter how reasonable the amount of compression you are claiming (at least to a degree), the fact you can claim this for already compressed data (avi and mp3) makes it highly suspicious that your assumptions are incorrect. Off hand the only method that this could be achieved is via look up patterns where all the patterns are stored on a users computer. This would allow you to encode the file much smaller, but in turn would require vast amounts of look up tables on the users PC, probably gigabytes. Which is just not a practical method.
Another problem is you are saying all the same things the guy in that thread said to defend his theory. Later he found various problems with his tests and eventually that it just didn’t work at all.
Don’t get me wrong though, I don’t know enough about Information theory and haven’t messed around with compression enough to ‘know’ you are wrong, just that I suspect you are missing something. So i’m certainly not saying to stop working on this, just that before investing too much time, be sure you know what you are doing.
Like I said that thread is good for some LOLS, but is also great for some very useful details on information theory and pointing out what has been tried in the past. Its just a damn good read.
However i’m not too proud to admit when I’m wrong, so when you’ve implemented this, please send me a beta and i’ll test it for you. When I discover you have found this algorithm that beats all existing ones i’ll publicly admit my error, in return for 0.01% of all profits you make (the algorithm must be commercially sold/licensed). Looking forward to being a millionaire by this time next year
Sounds like an error in reasoning, but I could be wrong 
Particularly interested in knowing how your method compares to other formats [p.s. looks like ‘AVI’ isn’t actually a compression scheme, rather a container for a variety of schemes].
I assume that you are both encoding and decoding and are checking for bit accuracy? I also assume you’re trying with a large sample of different files, and some randomly generated content? If not, I strongly suggest you do so - writing a console app to do this will only take an hour or so and help detect a wide range of errors.
Thats true, I guess its even possible for an AVI container to contain an uncompressed format, though I doubt that Meltdown just happened across such a file.
@Meltdown: Here’s another test for you can your method be used on itself? Can you repeatedly compress your compressed file with your method and still get the same level of compression, or any level of compression? If you can then that is another factor for questioning the soundness of the algorithm as its just not possible, becuase otherwise you could keep compressing it down to almost nothing
Look at it another way, could you pack 10 of these 18 mb compressed to 11mb video files into a single 110mb file and re-compress it? Afterall you are already claiming/thinking that you can compress already random data (i.e the output from most if not all compressed file formats), so that 110mb file becomes 50mb, and so on? If you think about it in this way you’ll see its just not possible.
This page might be worth reading compression-faq. It explains the counting argument (also know as pigeon-hole principle) along with other common misconceptions.
I’ve tried a few picture files, a few movie files, and a couple of MP3s… but beyond that I haven’t tried much else. But due to it’s nature, the compression will pretty much work with any file, although as I said I haven’t tried it with enough variety yet.
No, that wouldn’t work
Are you sure? I only ask because in your OP you show that all existing algorithms are unable to further compress video or mp3 to any real degree, suggesting that the source data is to all intensive purposes already random. Yet your proposed method has no problem with this random data, which should mean it could easily be used on already compressed files and the method could be employed recursively on itself.
Anyway I tried to point out the suspicious nature of your claims from an information theory point of view, not that you are deliberately making false claims (which I don’t believe you are) point of view. You seem pretty committed to this, so i’ll just wait in anticipation for you to post your proof/demo.
In the meantime i’m going to read the infamous GameDev MP3 Beating compression thread again for a laugh.
Yes I am sure. Without short of explaining exactly how I do it, I can understand why it doesn’t make sense.
First thing I’ll do when I get the performance working better is post a video showing it actually working… and no I don’t have any video editing skills whatsoever Then I’ll take it from there.
Ok, well good luck.
Don’t forget to have solid proof, you’ll going to need to release the ‘decompressor’, but you can keep the compressor code. We’ll then send you files which you will compress and return back to us, which with your previously supplied decompresser will have to reconstitute the original file with no loss of data.