Modeling analog gear in software - how to deal with feedback loops?

[Fender19]

I am trying emulate an electronic circuit in software and I have come to a roadblock. Many electronic circuits require feedback in order to operate. Something like voltage drop across a transistor (Vbe) depends on instantaneous feedback across the transistor if it's driving a reactive load. If that feedback is delayed the electronic circuit can (and likely will) become unstable.

So, how is a circuit like this modeled in the digital world where we have discrete time intervals - i.e., where feedback can never be instantaneous? A delay of even 1 sample can cause overshoot that would not occur in the analog world.

I have used SPICE modeling which accurately computes the electronic circuit response via algorithm - so I know it CAN be done - but HOW is it done?

[Vokbuz]

feedback can never be instantaneous

It can. Treat circuit as a system of equations. It is always possible to solve it. Not always analytically, sometimes iteration methods required.

[Fender19]

feedback can never be instantaneous

Not always analytically, sometimes iteration methods required.

Yes, this is what I'm having a hard time grasping - iteration. Seems that attempting to emulate electronic feedback may require parts of the plugin DSP code to perhaps run several times in an iteration loop WITHIN each sample period? Then it becomes a question of how close is "close enough" in the convergence vs CPU load, etc. Seems this could get very tricky...

[Urs]

Code: Select all

while (error > accuracy_I_want)
{
    error = f(predicted_result) - predicted_result;
   predicted_result = zeroFindingAlgorithmOfChoice( predicted_result, error )
}

where the classical zeroFindingAlgorithmOfChoice is Newton Raphson, bisection or secant method, depending on whether Newton is guaranteed to work or not.

Note that error and predicted_result can be vectors.

[Z1202]

feedback can never be instantaneous

Not always analytically, sometimes iteration methods required.

Yes, this is what I'm having a hard time grasping - iteration. Seems that attempting to emulate electronic feedback may require parts of the plugin DSP code to perhaps run several times in an iteration loop WITHIN each sample period? Then it becomes a question of how close is "close enough" in the convergence vs CPU load, etc. Seems this could get very tricky...

You can do the iterations "naive" way, letting the signal go through the feedback loop several iterations within each sample. This would essentially be (IIRC) the "fixed point" iteration approach to solving the equation. The probably most generic way would be to simply think of solving the equations, as Vokbuz mentioned earlier, where lots of numerical methods exist in math for solving analytically unsolvable equations. The most popular one is probably the Newton-Raphson, mentioned by Urs. Answering your question about how many iterations should you make, this is not a very straightforward question, but it's also addressed in numerical methods theory.

There are some further considerations about cases where thinking purely in equation terms might be insufficient, e.g. described in section 3.13 of the VAFilterDesign book rev 2.1.0 (although this also can be thought in terms of how well your equations represent what's going on) , but those are mostly academic.

[Z1202]

Also, besides the errors originating from 1 sample delay in feedback, there are further considerations regarding modeling analog circuits. Particularly the choice of the integration scheme is often of key importance. Another topic is aliasing. I strongly suggest that you read the respective literature (as from your question I guess that you might have not).

[Richard_Synapse]

Yes, this is what I'm having a hard time grasping - iteration. Seems that attempting to emulate electronic feedback may require parts of the plugin DSP code to perhaps run several times in an iteration loop WITHIN each sample period? Then it becomes a question of how close is "close enough" in the convergence vs CPU load, etc. Seems this could get very tricky...

Yes, however you can precompute the results and store them in a lookup table, if the network is not too complex. This should save a lot of CPU, but will cost memory of course. Another way is to use a neural network to predict the future states, basically acting as a solver that does not require iteration. If you only care about results, you could also try a neural net to predict just the output directly. It is all situational though and depends on what exactly you are trying to model. A generic SPICE-type approach is not going to be efficient in most cases.

Richard

[fefanto]

Another way is to use a neural network to predict the future states, basically acting as a solver that does not require iteration. If you only care about results, you could also try a neural net to predict just the output directly.

Are you aware of any literature (papers, open source code, products) directly related to applying a neural net in realtime "in the process block" of an audio plugin?
I've seen some plugins making use of AI/ML but as far as I'm aware it's all in the "parameter tweak" area.

thanks!
Michelangelo

[Richard_Synapse]

Are you aware of any literature (papers, open source code, products) directly related to applying a neural net in realtime "in the process block" of an audio plugin?

thanks!
Michelangelo

Yes, in the DAFX '19 conference papers.

Richard

[matt42]

Also, generally, you can solve analytically for the linear version of the circuit. This can provide a reasonably close guess for the nonlinear circuit, so hopefully you dont need so many iterations.

I didn't know anyone was looking at neural networks, but the thought did enter my head once during an idle moment

[andy-cytomic]

The mathematical keyword to describe these systems is "implicit". This contrasts with "explicit" systems which the output at the current time slice does not depend on itself. Implicit systems are solved for everywhere, in fact a DF1 biquad is a solution to an implicit system of equations, but it can be made explicit because it is linear, and low enough order. The difficult bit is when the equations are both implicit, and non-linear in the implicit term, which is where you are with a BJT.

I would recommend first implementing a full brute force implicit solver for your equations - this will be needed to check your results for any alternative solution you come up with anyway. There is plenty of information around on how to do this, most circuit simulators like SPICE and QUCS use MNA systems to do this - you can also check your full solver against a circuit simulator to make sure it's right. Once that's in place you will have a greater understanding of what is going on, and you'll be ready to check out more efficient solutions. I recommend looking into the DK method in particular, but there are many other ways to solve these equations, and many good suggestions have already been given.

[mystran]

The following is copied verbatim from what I wrote on a private forum a while ago into another question related to using MNA to solve non-linear circuits (using Newton, but the same general idea applies for other strategies too):

Order your matrix in the following way: completely static stuff (resistors, etc) go first. I/O and simple integrators (eg. fixed caps) goes second. Dynamic stuff that doesn't need iteration (eg. pots, variable caps, whatever) goes third. Anything that needs to be iterated (eg. with Newton) goes last.

Now, if you use LU to solve the matrix, then we can take advantage of the fact that eliminating a column only adds the scaled pivot row to the rows below (assuming the usual top-to-bottom order). Because the order of additions doesn't matter, you can do this before you have added all the relevant values to the rows below.

So.. this means you can factor (and then throw away) the static rows before you even start the plugin. This alone will shrink your matrix quite a bit. Once you know the sampling rate, you add the relevant values for the fixed caps and factor those rows. At run-time, these rows will only need forward and backward substitution, but no factorisation work.

On per-sample basis, you then add the dynamic values to the remaining bottom-right block and factor and forward-substituting anything that doesn't need iteration. Then you again save this partially factored result, such that you can start each Newton iteration by simply adding the equivalent conductances to the matrix, then factoring and solving the bottom-rightmost iterated block from the partial result. Finally when done with iteration, you backward substitute for the integrators and output.

Sometimes this doesn't quite work and you need to keep an extra row somewhere for pivoting reasons, but that's fine.

In fact, THIS is why I'm such a huge advocate for using LU to solve these things: as long as the matrix is ordered correctly, you can move the factorisation work out of any loop whenever all the values of the pivot-row and -column are known outside the loop, even if the bottom-right block is still only partially built.

edit: As far as I can tell, this is basically similar to methods like DK, but rather than having to develop any fancy theory, we can simply choose an intelligent pivoting order and then split the outer-most factorisation loop into different stages.

[andy-cytomic]

...
edit: As far as I can tell, this is basically similar to methods like DK, but rather than having to develop any fancy theory, we can simply choose an intelligent pivoting order and then split the outer-most factorisation loop into different stages.

I wouldn't really call matrix optimisation via LU decomposition the DK Method. The DK Method is primarily a pre-computation of a MIMO (multiple input multiple output) non-linearity table so that you can solve the non-linear system in a single step. Although in the original papers on the topic they use a State Space representation, it's not necessary to do so.