Understanding Convolution - Laplace

PistolsAtDawn

This is my first post on Boards.ie so unfortunately i'm unable to upload an attachment which would make asking this question a little clearer, but hopefully someone on here can offer some advice.

As the title suggests I am trying to understand the "mechanics" of convolution.

My basic understanding:

Given two functions g1(t) and g2(t), I can use the convolution operator to create a third function g3(t), which is some combination of g1(t) and g2(t). I am assuming this could be useful for a wide range of purposes. Although I don't know of any specific uses at the moment.

From my book:

g3(t) = integral from -infinity to +infinity of g1( T )g2( t - T )dT

Please Note: The uppercase 'T' in the formula above represents the Greek symbol 'Tau'.

To begin with, we have two signals g1(t) and g2(t) and both are functions of the variable 't', prior to taking the integral we introduce this new variable 'T'.

My Questions:
(1) Is 'T' a variable or a constant?
(2) Does g1 now become a function of T? OR is g1 evaluated at 'T' during the integration which would make it a constant during the integration?
(3) We "delay" (shift) g2 by 'T', for the integration, why?
(4) We perform the integration with respect to "small" changes in T, why?

Any help appreciated, I am really lost on this one, thanks.

ZorbaTehZ

(1) T is the integrating variable.
(2) g3 is now a function of T
(3) Not sure what you mean---if you integrated g1(T)g2(t) then this is just an integration of g1 because T doesnt appear in g2 and if you integrate g1(T)g2(T) then you get a number not a function.
(4) Don't know what you mean here.

PistolsAtDawn

ZorbaTehZ wrote: »

(1) T is the integrating variable.
(2) g3 is now a function of T
(3) Not sure what you mean---if you integrated g1(T)g2(t) then this is just an integration of g1 because T doesnt appear in g2 and if you integrate g1(T)g2(T) then you get a number not a function.
(4) Don't know what you mean here.

ZorbaTehZ wrote: »

(1) T is the integrating variable.
(2) g3 is now a function of T
(3) Not sure what you mean---if you integrated g1(T)g2(t) then this is just an integration of g1 because T doesnt appear in g2 and if you integrate g1(T)g2(T) then you get a number not a function.
(4) Don't know what you mean here.

Thanks for the reply.

To expand on some of the points you made:

Ok so, we are in agreement that T is the integrating function and not a constant.

But I suppose I don't understand why we introduce this variable 'T' in the first place?

With regard to (3), You state that integrating g1(T)g2(T) would return a scalar and not a function and I get this since for each distinct value 'T' we would be simply be evaluating both functions (g1 & g2) and then taking the product. We would then sum across the entire domain and end up with a scalar value.

So my next question is; how come integrating g1(T)g2(t-T) produces any difference, since the function g2 has simply been shifted to the right (i.e. positive direction) by 'T' units?
In my eyes this would still leave me with a scalar.

ZorbaTehZ

PistolsAtDawn wrote: »

But I suppose I don't understand why we introduce this variable 'T' in the first place?

So your question is what is the motivation to consider convolution? Well I think if one was going to try to combine functions using an integral this seems to me to be a fairly natural way to do it. Conceptually you are somehow averaging one function with respect to another function.

PistolsAtDawn wrote: »

So my next question is; how come integrating g1(T)g2(t-T) produces any difference, since the function g2 has simply been shifted to the right (i.e. positive direction) by 'T' units?
In my eyes this would still leave me with a scalar.

Try example: let g1(t)=1 and g2(t)=exp(-t^2). Also your view of "shifted to the right" isnt entirely correct, try looking at images here

dlouth15

PistolsAtDawn wrote: »

Given two functions g1(t) and g2(t), I can use the convolution operator to create a third function g3(t), which is some combination of g1(t) and g2(t). I am assuming this could be useful for a wide range of purposes. Although I don't know of any specific uses at the moment.

One are that it is used for is the blurring of images in packages like Photoshop.

If you imagine you are writing a program to blur an image. What you want is points on the image "spread out" according to some function. It could be a Gaussian function or some other function like the hat function.

Consider the image and the blurring function to be one-dimensional to keep it simple. Let g1(x) represent the image and let g2(x) represent the blurring function. It could be a Gaussian curve or a hat function or whatever.

Now by looking at how the convolution is carried out, you may be able to see how the convolution of the two produces the required blurred one-dimensional image.

[latex]\displaystyle{g_{3}\left(x\right)=\int_{-\infty}^{\infty}g_{1}\left(X\right)g_{2}\left(x-X\right)\, dX}[/latex]

It might help to consider the image as just a single point and see how this point is spread out according to the blurring function. Then add in another point and so on.

PistolsAtDawn

dlouth15 wrote: »

One are that it is used for is the blurring of images in packages like Photoshop.

If you imagine you are writing a program to blur an image. What you want is points on the image "spread out" according to some function. It could be a Gaussian function or some other function like the hat function.

Consider the image and the blurring function to be one-dimensional to keep it simple. Let g1(x) represent the image and let g2(x) represent the blurring function. It could be a Gaussian curve or a hat function or whatever.

Now by looking at how the convolution is carried out, you may be able to see how the convolution of the two produces the required blurred one-dimensional image.

[latex]\displaystyle{g_{3}\left(x\right)=\int_{-\infty}^{\infty}g_{1}\left(X\right)g_{2}\left(x-X\right)\, dX}[/latex]

It might help to consider the image as just a single point and see how this point is spread out according to the blurring function. Then add in another point and so on.

Thanks for your help, i'll do a bit of work on this at the weekend and get back to you.

Michael Collins

PistolsAtDawn wrote: »

This is my first post on Boards.ie so unfortunately i'm unable to upload an attachment which would make asking this question a little clearer, but hopefully someone on here can offer some advice.

As the title suggests I am trying to understand the "mechanics" of convolution.

My basic understanding:

Given two functions g1(t) and g2(t), I can use the convolution operator to create a third function g3(t), which is some combination of g1(t) and g2(t). I am assuming this could be useful for a wide range of purposes. Although I don't know of any specific uses at the moment.

From my book:

g3(t) = integral from -infinity to +infinity of g1( T )g2( t - T )dT

Please Note: The uppercase 'T' in the formula above represents the Greek symbol 'Tau'.

To begin with, we have two signals g1(t) and g2(t) and both are functions of the variable 't', prior to taking the integral we introduce this new variable 'T'.

My Questions:
(1) Is 'T' a variable or a constant?
(2) Does g1 now become a function of T? OR is g1 evaluated at 'T' during the integration which would make it a constant during the integration?
(3) We "delay" (shift) g2 by 'T', for the integration, why?
(4) We perform the integration with respect to "small" changes in T, why?

Any help appreciated, I am really lost on this one, thanks.

Convolution is most commonly used as a method for determining the output of a system when you know the input. It is very important to get this!

I'm unsure how much you know about the topic of "systems", but if you're studying the Laplace transform I guess you have come across the idea. I'll try to give a brief explanation here anyway. So here goes:

Let's say we have an input x(t) to a system. This system could be any of a number of different things. Let's say it's a car. And let's say x(t) represents the force you apply to the accelerator in the car. Now let's consider the output of this system, as the speed of the car. Call this y(t).

Clearly there is a relationship between how hard you push the accelerator and the resulting speed of the car. This relation can be described by the "impulse response", let's call this h(t). This is a property of the system only, it describes the system i.e. the car.

The beauty of convolution is that, using the function h(t), we can determine the output y(t) i.e. the speed, for any given input x(t), according to your formula:

[latex] \displaystyle y(t) = \int_{-\infty}^{\infty} h(T) x(t-T) dT [/latex]

i.e. to get the output y(t) we simply "convolve" the input x(t) with the impulse response y(t). It's hard to see *how* this works using continuous signals like this, discrete convolution is easier to understand first.

This might help: https://www.youtube.com/watch?v=yzV3xW8YfzQ

Also check out this video: https://www.youtube.com/watch?v=zoRJZDiPGds

Now to answer your questions:

(1): T is what is known as a dummy variable. It is just used to shift one of the functions during the convolution, it could be given any symbol.

(2) g1(T) is a function of T - you call tell this because the integration is with respect to T, so T must change!

(3) The delay is because any system might have "memory". In the car example the speed of the car now, might depend on how much force was applied to the accelerator 10 seconds ago, so the convolution takes this into account by shifting the input back in time.

(4) Integration is always performed as small changes. We have to add up the effect of all the past inputs, this is what integration does i the convolution integral.

The really nice thing here is when you use the Laplace transform. Hopefully it makes sense to you now that convolution is very useful; it can tell us the output for any input. But unfortunately it is hard! Calculating the integral can be very time consuming...

The Laplace transform makes this much simpler - but that's the topic of another post.

Feel free to post back any questions you might have about this or the videos above.

PistolsAtDawn

Michael Collins wrote: »

Convolution is most commonly used as a method for determining the output of a system when you know the input. It is very important to get this!

Thanks very much for the very informative post. I only have a few spare minutes at the moment, but i'll try to give you a brief summary of my current understanding derived from yours and the previous posts. (Note: I have not got a chance to watch those videos yet, as I stated earlier I will have some time to delve into this further over the weekend so I will probably know a little more by Sunday/Monday).

From your system example.

I understand that Convolution provides us with a method to determine the output y(t) for any input x(t) using the function h(t).

h(t) {transfer function?} describes the system of interest and it is known to us since we are either designing the system or have access to its transfer function.

The beauty of convolution comes from the fact that when we are designing the system (a car) in this case; we need to know how the drivers interaction with the accelerator results in car speed.

My understanding of the "mechanics" of the convolution integral shown below, are as follows:

(1) We throw in a dummy variable T, so that now the system input (i.e. the signal sent from the accelerator to the engine) is a function of T.

(2) We also sub in the dummy variable T to the transfer function h(T) but we flip this around the y-axis as follows h(-T)

(3) We now use the variable 't' to perform a shift on our "flipped" transfer function by some amount which we deem appropriate for the purposes of our evaluation.

(4) We then take the product of x(T) and h(t-T) from -infinity to +infinity (Really it is only necessary to perform the integration over the domain which the functions are defined as it will result in zero everywhere else since (a)(0)=0, where a is constant.)

(5) The output y(t) is then the sum of the product between the system response and input.
The reason for the summation is because we want to know the previous inputs because the driver could have been travelling at 60kmph and then presses the accelerator further, increasing the magnitude of x(t), in turn increasing speed to say 80kmph. Therefore we need to know previous input to determine correct output.

This then leads me down the route of choosing the correct value of 't' to shift the system transfer function accordingly.

Apologies if this post is a bit rag tag, I only have a few spare minutes here so no time to go over it, just typing as I think about what you said, i'll clean it up later. Thanks again, I feel I am starting to get it. But I suppose you guys will be the judge of that. :-)

Michael Collins

PistolsAtDawn wrote: »

Thanks very much for the very informative post. I only have a few spare minutes at the moment, but i'll try to give you a brief summary of my current understanding derived from yours and the previous posts. (Note: I have not got a chance to watch those videos yet, as I stated earlier I will have some time to delve into this further over the weekend so I will probably know a little more by Sunday/Monday).

From your system example.

I understand that Convolution provides us with a method to determine the output y(t) for any input x(t) using the function h(t).

h(t) {transfer function?} describes the system of interest and it is known to us since we are either designing the system or have access to its transfer function.

Almost, h(t) is the "impulse response" and is related to the transfer function of the system through the Laplace transform i.e. H(s) (the transfer function) is the Laplace transform of h(t)

The beauty of convolution comes from the fact that when we are designing the system (a car) in this case; we need to know how the drivers interaction with the accelerator results in car speed.

My understanding of the "mechanics" of the convolution integral shown below, are as follows:

(1) We throw in a dummy variable T, so that now the system input (i.e. the signal sent from the accelerator to the engine) is a function of T.

Exactly.

(2) We also sub in the dummy variable T to the transfer function h(T) but we flip this around the y-axis as follows h(-T)

Yep!

(3) We now use the variable 't' to perform a shift on our "flipped" transfer function by some amount which we deem appropriate for the purposes of our evaluation.

Perfect!

(4) We then take the product of x(T) and h(t-T) from -infinity to +infinity (Really it is only necessary to perform the integration over the domain which the functions are defined as it will result in zero everywhere else since (a)(0)=0, where a is constant.)

Yes, the product is integrated over the range for which both functions are not zero.

(5) The output y(t) is then the sum of the product between the system response and input.
The reason for the summation is because we want to know the previous inputs because the driver could have been travelling at 60kmph and then presses the accelerator further, increasing the magnitude of x(t), in turn increasing speed to say 80kmph. Therefore we need to know previous input to determine correct output.

The previous inputs are important since most systems have memory. In the car example the system intuitvely must have some memory. This is because the speed of the car at any instant t won't just depend on what the present input is (how far down the accelerator is pushed), but you need to know what the car has been doing recently as well, i.e. how far down was the accelerator 1 second ago. Or 5 seconds ago.

For example:

if a car was stopped 5 seconds ago, and now the acclerator is all the way down, it's unlikely the present speed is the max speed the car can do.

while if the car was already at close to max speed, then maybe 5 seconds later it would have reached its actual maximum -- the car system has memory. In electronics, inductors and capacitors have memory. In a mechanical system, springs have memory. So many systems have memory!

This then leads me down the route of choosing the correct value of 't' to shift the system transfer function accordingly.

Not quite sure what you mean here - there is no "correct" value of t as such. The variable t is just that, a variable, that tells you what the output y(t) is at time t.

Apologies if this post is a bit rag tag, I only have a few spare minutes here so no time to go over it, just typing as I think about what you said, i'll clean it up later. Thanks again, I feel I am starting to get it. But I suppose you guys will be the judge of that. :-)

You're definately getting there!

If you really want to understand why convolution works i.e. why the formula is the way it is, having a look at discrete convolution is the way to go in my book.

PistolsAtDawn

Right, thanks again for your help. I had a think over the weekend about what you said in post #7.

I feel i'm becoming more comfortable with the "mechanics" of convolution. In 'Section 1' below I will give a description of the mathematics from my current viewpoint. 'Section 2' will hopefully describe my physical interpretation of the convolution problem.

Note: Questions will be noted during the text as follows; (->Qn<-), where 'n; represents 1,2,3..., the questions themselves can be found underneath section 2.


Section 1:
Given some input x(t) and a system impulse response h(t); we can apply the convolution transformation to find y(t) = x(t)*h(t)

To do this we find; Integral [x(T) h(t-T)] over the "RANGE" for which both are defined, did you mean domain/independent variable here? (Expansion of question; you stated range so I am assuming we integrate over t but only for the values of t which produce an output in either function, is this what you meant?)

(1) First we must mirror h(t) about the y-axis as follows; h(-t),
Considering a function h(t) where h(t) = 0 (for t<0),
h(t) = t (for 0<t<t1),
h(t) = 1 (for t1<t).
We find that h(-t) = 1 (for t<-t1),
h(-t) = -t (for -t1<t<0),
h(-t) = 0 (for 0<t).

We can introduce a dummy variable 'T' = 't' so that h(-t) becomes h(-T).

(2) We can also shift this function h(-T) along the t-axis by applying a shift of 't' as follows; h(-T) -> h(t-T)
This means that by selecting a value for 't', for example t = t1, the above function h(-T) becomes h(t1-T) which results in a left-shift/negative
direction shift of the function/shift the function back in time (i.e. h(t1-T) = 1 (for t<(-2t1), h(t1-T) = -t (for -2t1<t<t1) and h(t1-T) = 0 (for t1<t).


(3) Anyway, now that we have established the new form of 'h' we can move on to the process of finding the convolution;
Basically for any time 't' the convolved signal y(t) will be the result of a sum of products. The product terms represent the input magnitude at time 't' and the impulse response at time 't-T'.

(4) By taking infinitesimally small steps 'dT' along the domain and calculating the product between x(t) and h(t-T) and then summing up all the results we get an area. This area, in general terms, represents (->Q1<-), and is represented by the function y(t).



Section 2:
The problem at hand relates to the design of an 'accelerator system' so we start with some input signal x(t), for example; force applied to car accelerator. This signal would most probably be the output from some sensing device that senses accelerator pedal position and transfers this into a representative electrical signal. We feed this signal into a 'speed controller' whose impulse response h(t), a description of the systems response to a short-duration time-domain signal (->Q2<-), is known to us. The output of this controller is a signal y(t) which will be a signal that 'commands' the car engine to increase power which results in an increased speed. (I won't get into feedback and the actual input to the "plant" being the difference between x(t) and y(t) sometimes referred to as the error e(t)...we'll leave that for another day)

Since we are world class electronic systems designers :-) we understand that in order to ascertain current speed we must possess knowledge of our previous speed(s). Here is where I get a little lost on the practical side of things; what information is the convolution of the signal and the kernal h(t-T) providing us. I can visualize h(t-T) "sliding" over x(T) and performing the multiplication and then summation, but I am lost as to what this new information represents in this practical example. To my mind the x(T) contains information all input information and h(T) represents ____???__


(->Q1<-) In general/laymans terms what does the convolved function represent?

(->Q2<-): This notion of impulse response is escaping me, what is the relevance of a 'short-duration time domain signal', i.e. Dirac.
Why do we care about the response to such brief signals, conversely; why are we not interested in long-duration time- domain signal response?

Thanks again for your help , sorry about all the questions and reiteration, as you have probably guessed Mathematics is not my forte.