A new language

DSP, Plugin and Host development discussion.
RELATED
PRODUCTS

Post

All the Reason Refill audio is also lossless even though it's packed when it's on your harddrive. Are you telling then that while graphix are pixels in row the audio isn't quiet the same in that manner then?

Post

No amount of trickiness can work around certain limits; it's not just infeasible, it's not possible at all.

Really, read the two links paterpeter posted previously (ha ha). Understanding the Nyquist-Shannon sampling theorem, etc. is absolutely required to make any headway on this topic.

Post

Meffy wrote: Mon Jan 20, 2020 11:38 pm it's not possible at all.
It might be possible for the graphics and sending stuff unrelated to the audio. Anyway, how do the Propellerheads do it then? The Props have done it already 20 years ago.

Anyway, isn't audio also based on the digital data in the same manner as graphics are in pixels then?

Post

deep'n'dark wrote: Tue Jan 21, 2020 12:43 pm
Anyway, isn't audio also based on the digital data in the same manner as graphics are in pixels then?
Absolutely. Lossy algorithms used to reduce the size of either kind of media (jpg, mp3) makes use of the assumption that we can drop parts of the data (mostly high frequencies) without our senses (ears, eyes) noticing too much (plus some clever encoding tricks, quantization, dropping redundancies etc).
This assumption does not hold for all kinds of data. You can't drop letters from an email without anybody noticing it very quickly.

As a general purpose compression method, lossy algorithms such as mp3 just won't work well.

Post

If you have a song/pattern that consists of a small set of samples, you could "compress" it by having the sample data and the instructions (like MIDI messages) to play those samples. (And if it's purely synthesis based, you could even drop the audio samples and have just the playback instructions data.)

But if you have something like an acoustic recording of a Beethoven symphony, each moment in that is unique and can't be repeated to reproduce the original data. The only way to compress something like that losslessly is to use techniques like used in FLAC and you can expect around 50% reduction in the audio data size.

For other kinds of data like text, raster images and binary executables, there are well researched and established techniques for losslessly compressing them. I don't expect any new revolutions to happen with those.

Post

Ok, thanks folks for making it clear. How about DirectX or Java or any other languages - would anything at all benefit from my idea. Probably not. :P

Post

Your idea isn't bad, it's just that there are limits to how much a given signal (audio, graphics, text, doesn't matter what, once digitized they're all just streams of bits) can be compressed without losing so much information that the result sounds or looks or reads wrong.

For an easy demonstration, load a detailed photo into Photoshop or some other image editor. Try saving the photograph as .jpg many times with different compression settings, from least compression to most. You'll see the file size drop as image quality decreases. For the worst settings you get absolutely horrible results, while for the best quality the output might be nearly indistinguishable from the original — though comparing them pixel by pixel will show changes, the eye can't pick them out.

JPEG is an example of lossy compression. The output loses information that was in the original. You can crunch an image down to very small file size if you're willing to lose a lot of image quality.

Lossless compression provides output that exactly reproduces the input, with no change. An example is run length encoding, or RLE. Imagine a 100x100-pixel image, mostly a single shade of red with one green pixel. All the red pixels (a "run") that come before the green one can be stored as a simple count (how many red pixels) and a number storing the shade of red. Then you store one green pixel, then another count-and-color entry telling how many red pixels to the end. Very strong compression, but the output is identical to the input. The drawback is that most useful images aren't so simple, so compression isn't as dramatic as in the red-green-red example. If no two successive pixels are the same, there are no "runs" of the same color, RLE compression won't compress the image but make it larger by one byte per pixel. (In such cases you just use the uncompressed image.)

To apply this to audio, RLE could compress a sample of mostly silence with a short blip of sound by turning the silence into a run of zeroes, then storing the non-silent part as plain audio data.

RLE is conceptually simple. JPEG image compression, and the audio compression used in MP3 and similar methods, are much trickier. These methods involve extracting enough data to resynthesize a reasonable (you decide how reasonable when you choose the amount of compression) simulation of the original data. More compression, less fidelity.

Every once in a while new compression methods are devised, some of which provide really impressive compression without much noticeable degradation of the input signal. Here the trade-off isn't just quality versus output size, but also quality versus time to compress or decompress. An example is JPEG2000, which is great for squashing documents scanned at high resolution, such as maps. High compression levels can be used without totally wrecking the image as JPEG would. Why doesn't everyone use JPEG2000 compression? It's very slow compared to common JPEG, and anyone using it must obtain a license (though the patent holders have agreed to issue these free of charge).

So while it's not impossible to compress digital signals to very small output files, there are always trade-offs involved, and the better the compression, the more complex the code needed to do the job (and, as a rule of thumb, the harder it is to understand the compression algorithms).

Don't let this discourage you from experimenting, though. Some of the best lessons I've had were attempts to do something I'd been told wasn't possible. I failed too, but I learned a lot by trying — and things you learn by doing are the best-learned lessons of all.

[eta] I've simplified the descriptions above. Real-world RLE implementations, such as GIF and PNG, are slightly more complicated. Still, conceptually they're far simpler than lossy compression methods.

Post

@Meffy - thanks! I need to re-read your reply more than once.

The meaning thing, I was even thinking how The Inspector VST and such plugins also meter data, so if something is different in a mp3 stem in the maximum dense, one could stream it and render data from the mp3.

Post

Tip: As I have heard, if you are trying to bypass the pigeonhole principle to encode information: Use transforms to encode the data, not to reduce data. As was stated before, you cannot reduce beyond a given size, but you can transform upon a fixed size an infinite number of times.
SLH - Yes, I am a woman, deal with it.

Post

*salute*

I might have got some points wrong, and my writing isn't entirely clear, so if anyone finds a mistake please don't hesitate to correct it. Also, I left out things such as headers, color lookup tables, and so on. I also should have had at least two-byte counts in my RLE example, or store each row of pixels separately, and… you get the picture (so to speak), I concentrated on the raw image data while omitting lots of details that might have confused the central issue.

If you've not tried programming, it might be worthwhile to learn Python or some other modern programming language. That way you can try out ideas using "toy data" as input — data you created specifically to feed into your experimental data-squeezer, data that you know everything about 'cos you crafted it by hand.

Hope you enjoy the research!

Post

Vertion wrote: Wed Jan 22, 2020 3:06 am Tip: As I have heard, if you are trying to bypass the pigeonhole principle to encode information: Use transforms to encode the data, not to reduce data. As was stated before, you cannot reduce beyond a given size, but you can transform upon a fixed size an infinite number of times.
Basically all types of entropy coding work by using some kind of variable bit-length representations such that the more probable a given input bit-string, the shorter the output pattern assigned to it.
This then hopefully leads to compression with "probable" (ie. "low entropy") strings, but also will always result in expansion with "improbable" (ie. "high entropy") strings.

The way content specific lossless compressors usually work is that they try to transform the data in such a way as to make it more predictable, then sometimes remove the predictable part and just keep the "error" signal (because we can predict again at the decompression stage), then use the entropy coding to actually pack the remaining information into a (hopefully) smaller file.

It is important to remember that this is always worse than zero sum: there will always be inputs that result in expansion rather than compression and the best you can do is detect these cases and just store them "as-is" instead (which is technically also expansion, because you always need at least an extra bit to indicate you didn't use any actual compression).

Post

So if the VST checker would check out the amplitude, panning position etc of the minimal diversity, you could extract stuff form the mp3 like hell. All you'd need is an software that reads the file and extracts the data.

Post

Let's put it this way: Sizeable amounts of very clever people trained for years have worked on every technical field trying to push the boundaries, lots of them with real passion.

You are not going to beat them untrained without first reaching their level, at least not on signal processing. It's not that easy.

Post

deep'n'dark wrote: Mon Jan 20, 2020 7:06 pm This is my vision about a new computer language:
So lets assume you'd slice one minute long audio into as many slices as you can. Then you'd make a meaning for to every different kind of audio slice. As example slice1 would mean A in a binary, another slice B etc. Anyway, if you'd pack 10 000 slices into an one minute long mp3 128kb/s, you then could extract this mp3 into any kind of digital data. The magic is that 1 minute long mp3 is smaller in size than the extracted data afterwards. Possible?
This thread s from quite a while ago, but it seems youve revived it, so I'll just ask...

If we ignore the data compression/translation bits, what about this is a 'new computer language?'

Going to the data compression/translation bit, your method reads as follows:
  • Encode 8-bit characters to 16-bit audio samples
  • Modify the 16-bit audio sample by putting it through a lossy codec
  • Reconstruct that to a 16-bit audio sample, which will be different from the original
  • Decode that, somehow getting back the original 8-bit character

    Is that correct?
my other modular synth is a bugbrand

Post

whyterabbyt wrote: Sun Nov 08, 2020 12:31 pm
deep'n'dark wrote: Mon Jan 20, 2020 7:06 pm This is my vision about a new computer language:
So lets assume you'd slice one minute long audio into as many slices as you can. Then you'd make a meaning for to every different kind of audio slice. As example slice1 would mean A in a binary, another slice B etc. Anyway, if you'd pack 10 000 slices into an one minute long mp3 128kb/s, you then could extract this mp3 into any kind of digital data. The magic is that 1 minute long mp3 is smaller in size than the extracted data afterwards. Possible?
This thread s from quite a while ago, but it seems youve revived it, so I'll just ask...

If we ignore the data compression/translation bits, what about this is a 'new computer language?'

Going to the data compression/translation bit, your method reads as follows:
  • Encode 8-bit characters to 16-bit audio samples
  • Modify the 16-bit audio sample by putting it through a lossy codec
  • Reconstruct that to a 16-bit audio sample, which will be different from the original
  • Decode that, somehow getting back the original 8-bit character

    Is that correct?
I didn't bump this thread up myself.

Post Reply

Return to “DSP and Plugin Development”