Sonogram palettes?

[alex.g]

Hi fellow developers !

This is my first post here, although I've been reading around this useful knowledge base (which is kvr) for some time...
I could introduce myself as a graduating computer science student, who likes computer music and guitar playing; i've been studying DSP development in my spare time for two years, in particular referred to the VST platform programmed in C++, and I do some "home audio processing" since 1996.
I think I should say thank to all of you, since a good part of what I've learned about plugins programming comes from this forum:).

But let's go on with my question...

These days i'm designing a Sonogram / Spectrogram (obviously GUI) plugin.
That's to say, audio input is represented as a 2D plot showing frequency power related to time by using a coloured scale.
I know about a couple of palettes used in audio analysis (please correct me if I'm wrong):

• - Fade from black (-oo) to light green (0 dB);
  - Fade from violet (-oo) to red (0 dB) using an inverted rainbow prism light.

Anyone knows about any other useful scale used in scientific analysis?
I'll give the user the possibility to create custom palettes, but I'd like to put some handy professional scales, presetted in the GUI.

Thank you by now, I wish a lot of creativity to everybody!
Alex

[MadBrain]

I've seen a Buzz spectrogram (Geonik's visualisation) offer two settings - black to white and rainbow.

One thing I'd personally like to see is amplitude as brightness and phase difference as color.

[alex.g]

One thing I'd personally like to see is amplitude as brightness and phase difference as color.

I didn't think about plotting phases...
Perhaps these could be represented with a "circular" palette, in order to have a stepless hue around +/-PI, but... I've never seen an "absolute" phase analysis, just a phase difference between two stereo channels for example.
If you have some interesting example demonstrating it could be useful, please make me know, I'm really curious !

Thank you for your suggestion about the brightness.

[Borogove]

Wow, a light just went on in my head. In an FFT bin, you don't really know what the exact freqs represented are, just a range. But if you plotted phase as color, then a slightly lower frequency within the bin would "red-shift" as you read across the sonogram, and a slightly higher one would "blue-shift". I think. Maybe. Possibly. Or I could be smoking crack.

[MadBrain]

Wow, a light just went on in my head. In an FFT bin, you don't really know what the exact freqs represented are, just a range. But if you plotted phase as color, then a slightly lower frequency within the bin would "red-shift" as you read across the sonogram, and a slightly higher one would "blue-shift". I think. Maybe. Possibly. Or I could be smoking crack.

Exactly! You could also use the phase change from bin to bin instead of the raw phase, so that instead of reading the color cycling speed, you can see the exact frequency just by looking at the color. That would also mean that constant tones (even non-harmonic ones) would have a constant look, while noise or quick phase transitions would be easy to see, even if the amplitude stays constant.

[alex.g]

This is getting interesting...
Plotting a coloured first derivative (let's call it discrete difference, it's nicer!) could be useful to find anomalies or hidden noises, right? I'll do some experimenting.
I'm curious about the shift thing, Borogove, I'll check it out.

There's always the issue with the +/-PI phase wraparound. If I remember well, if we have a bin with a value of 3.11° and the following with -3.01°, much probably the difference is only 0.24 degrees, instead of almost a full period. Should I consider the minimum of the two possible differences?

[Borogove]

There's always the issue with the +/-PI phase wraparound. If I remember well, if we have a bin with a value of 3.11° and the following with -3.01°, much probably the difference is only 0.24 degrees, instead of almost a full period.

If you use a HSV (Hue/Saturation/Value) color space, with phase mapped to hue, this is taken care of. (Actually, it just means that the problem moves to the color space conversion; I don't know of a really fast technique for HSV->RGB.)

[MadBrain]

Yeah, it would look something like this:

Code: Select all

   0° = red
+ 60° = yellow
+120° = green
±180° = cyan
-120° = blue
- 60° = magenta

As for fast HSV conversion, I'd do it with a LUT, with some compensations for things like the brightness difference (yellow is much brighter than blue) and with emphasis on some parts of the spectrum (usually it's easier to see the different shades in the red-yellow part of the spectrum than the green-cyan for instance).

[alex.g]

In fact I'm implementing it by using a LUT.
For the power spectrum colors I'm using a 16 bit (65536 levels) palette.
Being input values between 0.0 and 1.0 included, I scale them to logarithmic, then I map them to the output image as follows:

Code: Select all

out[i] = palette[(int)(in[i] * 65535.F)];

Every color used for the output is already there, just a multiplication and a indirect memory access are involved for each bin evaluated.
The palette is generated at load time; for example, to get a fading scale between black and green, I write something like this:

Code: Select all

CSoundPalette palette;
palette->addColor(0x00000000, 0.F);
palette->addColor(0x0000FF00, 1.F);

If I wanted a red zone around middle power, I'd put:

Code: Select all

palette->addColor(0x00FF0000, 0.5F);

The object itself fills the palette array by fading adiacent colors with an equal power function.
I'd like to know more about brightness compensations, could you suggest me something, MadBrain?

[MadBrain]

Okay, this is pretty nasty code, slow and not protected, and in fact I dunno if it'll even compile, but it should give an idea I think...

It works by determining the brightness of the pure saturated color (ie bright red, green, etc...) and then if it's lower it darkens(multiply) the color and it it's higher it lightens it ("screen" in photoshop, basically mix with white).

Code: Select all

typedef struct
{
	float r,g,b;
} color;

color table[] =
{
	{1,0,0},{1,1,0},{0,1,0},{0,1,1},{0,0,1},{1,0,1},{1,0,0},{1,1,0}
};

float intensity[] =
{
	0.33,0.83,0.5,0.67,0.17,0.5,0.33,0.83
};

color get_color(float brightness, float hue)	// brightness and hue must be 0..1
{
	int q_hue = hue;
	float f_hue = hue - q_hue;
	float bright_lim;
	color temp;

	bright_lim = intensity[hue]*(1-f_hue) + intensity[hue+1]*f_hue;

	if(brightness > bright_lim );
	{
		brightness = (1 - brightness) / (1 - bright_lim);
		temp.r = table[hue].r*(1-f_hue) + table[hue+1].r*f_hue;
		temp.g = table[hue].g*(1-f_hue) + table[hue+1].g*f_hue;
		temp.b = table[hue].b*(1-f_hue) + table[hue+1].b*f_hue;

		temp.r = 1 - temp.r;
		temp.g = 1 - temp.g;
		temp.b = 1 - temp.b;

		temp.r *= brightness;
		temp.g *= brightness;
		temp.b *= brightness;

		temp.r = 1 - temp.r;
		temp.g = 1 - temp.g;
		temp.b = 1 - temp.b;

		return temp;
	}
	else
	{
		brightness = (brightness) / (bright_lim);

		temp.r = table[hue].r*(1-f_hue) + table[hue+1].r*f_hue;
		temp.g = table[hue].g*(1-f_hue) + table[hue+1].g*f_hue;
		temp.b = table[hue].b*(1-f_hue) + table[hue+1].b*f_hue;

		temp.r *= brightness;
		temp.g *= brightness;
		temp.b *= brightness;

		return temp;
	}
}

[alex.g]

Ok, I don't know much about screening and darkening theory, but your example is quite clear to understand, then it will surely be useful once I work over it.
By now, I'm preparing a basic prototype by using RGB color space applied to power spectrum. This because it's simpler, and I want to test its speed first. Once I'll have this point clear, I'm planning to try out the HSL conversion for the phase information.

The problem will be memory. By now I need 8MB, just for the internal representation of power spectrum history, plus 1MB for the final image to be blitted (512x512 pixels 32bit). This could sound strange but it's related to detail/performance issues, I can assure it makes sense!
If I put a phase analysis in the same plugin, I've to keep in count 8MB more.
Well, if I manage to get it good, this will be an affordable requirement. And a nice feature!