I'm wondering, if a cloud service does all audio processing and you only need to a design a plugin (vst, audiounit, aax) that communicates with that cloud service and uploads/downloads audio, how complicated is that? I'm a developer but I only did C++ in college, so JUCE is a bit overwhelming.
I would want ARA integrated, which i imagine makes it slightly tricker, but is this overall idea pretty simple?
Alternative is to hire someone to code this, but finding a good JUCE developer a little more difficult, and if this is as simple of an idea as it seems, I'm just wondering if i could tackle it easily myself.
JUCE best platform for cloud processing plugin?
-
- KVRer
- Topic Starter
- 3 posts since 26 May, 2015
-
- KVRAF
- 15313 posts since 8 Mar, 2005 from Utrecht, Holland
A plugin is supposed to work in (nearly) true real-time.
Fetching data from a network resource requires a wait of unknown amount. So buffering and latency makes it complex.
Fetching data from a network resource requires a wait of unknown amount. So buffering and latency makes it complex.
We are the KVR collective. Resistance is futile. You will be assimilated. 
My MusicCalc is served over https!!
My MusicCalc is served over https!!
-
- KVRAF
- 7940 posts since 12 Feb, 2006 from Helsinki, Finland
What BertKoor says is relevant if you want to actually try streaming audio on the fly... which sounds like "less than ideal" concept (eg. cloud gaming can tolerate a bit more latency and that's not working very well either).
But.. assuming we don't care about latency, you can basically create a TCP socket, connect it to the service and then you have a bidirectional "pipe" where you write stuff and the server writes stuff back. There are various libraries to handle the dirty details of networking protocols for you, but even using just a TCP socket, something like HTTP is actually reasonably simple to use.. which is probably why it's kinda popular. Unfortunately if you want to talk HTTPS (ie. with TLS) then you obviously need to handle the whole TLS layer and you're probably best off just using a library that can handle all that stuff for you.
If you do plan on streaming audio though (and are willing to add enough latency to deal with the realities of the internet) then you probably want to build a custom protocol over UDP though. With UDP you send a packet, it gets wrapped into an IP packet and it might or might not arrive at the destination, in whatever order with respect to other packets you send. This means that you also need a strategy for dealing with lost data, perhaps by sending redundant information or whatever, but being able to choose what to do is great when you're dealing with latency sensitive stuff, because if TCP (which guarantees in-order "stream" delivery) ever loses a packet, it needs to stop everything and wait for redelivery... which (1) happens all the time in the real internet and (2) is a total no-go for anything that is latency sensitive.
There's nothing very fundamentally different between writing a network application and writing a plugin that talks over the network, but network programming itself is it's own special topic which does have some amount of learning curve. There are tutorials around though... and if you DO plan on doing something latency sensitive, the gamedev community has good resources for the whole "how to pretend internet can actually handle realtime" thing (eg. the basic networking stuff from Gaffer on Games that isn't specific to game stuff as such could be useful).
But.. assuming we don't care about latency, you can basically create a TCP socket, connect it to the service and then you have a bidirectional "pipe" where you write stuff and the server writes stuff back. There are various libraries to handle the dirty details of networking protocols for you, but even using just a TCP socket, something like HTTP is actually reasonably simple to use.. which is probably why it's kinda popular. Unfortunately if you want to talk HTTPS (ie. with TLS) then you obviously need to handle the whole TLS layer and you're probably best off just using a library that can handle all that stuff for you.
If you do plan on streaming audio though (and are willing to add enough latency to deal with the realities of the internet) then you probably want to build a custom protocol over UDP though. With UDP you send a packet, it gets wrapped into an IP packet and it might or might not arrive at the destination, in whatever order with respect to other packets you send. This means that you also need a strategy for dealing with lost data, perhaps by sending redundant information or whatever, but being able to choose what to do is great when you're dealing with latency sensitive stuff, because if TCP (which guarantees in-order "stream" delivery) ever loses a packet, it needs to stop everything and wait for redelivery... which (1) happens all the time in the real internet and (2) is a total no-go for anything that is latency sensitive.
There's nothing very fundamentally different between writing a network application and writing a plugin that talks over the network, but network programming itself is it's own special topic which does have some amount of learning curve. There are tutorials around though... and if you DO plan on doing something latency sensitive, the gamedev community has good resources for the whole "how to pretend internet can actually handle realtime" thing (eg. the basic networking stuff from Gaffer on Games that isn't specific to game stuff as such could be useful).
-
- KVRer
- Topic Starter
- 3 posts since 26 May, 2015
Ah sorry this is not for realtime processing. This is similar to vocalign or melodyne, where I’d want to be able to transfer audio into the plugin, the plugin then uploads the audio to a server where it gets processed, and then when it’s done the plugin downloads the audio and plays it in place of the original audio.
