Melodyne DNA

JJV

Got an e-mail from Celemony offering the chance to download the beta.

Has anyone had the chance to check this out yet? I was hoping to get a couple of hours to play with it over the weekend.

The guys on the Sonicstate podcast seem very impressed, going so far as to call it revolutionary?

http://www.sonicstate.com/news/

www.jjvernon.com

woodsdenis

I am sure this technology will be looked back on as a real game changer.
The ability to manipulate mixed files like this is revolutionary. Does it work
perfectly now, absolutely not, but this is just the beginning.

PaulBrewer

But how da feck does it work ?

krd

PaulBrewer wrote: »

But how da feck does it work ?

That I do know.

There is many "qualities" to a sound - The one you're used to seeing when you look at a sound wave in a sound wave editor is Amplitude and Frequency of a Sine wave over time.

But there is a "third" quality that can recently be identified by software that is "timbre"

A guitar or piano - or voice - will have a characteristic "timbre" - The human ear can hear and separately perceive that - up to now computer software has been really bad at that kind of thing as it tends to view the waves in absolute mathematical terms. (Frequently not seeing the wood for the trees)

If you do a search on the web you'll find pictures of sound analysis that show timbre - in 3d so the sounds can be separated.

(I met someone years ago who was doing a Phd in this kind of thing - The human ear can perceive a drummer playing drums - and can focus in on the drummer and perceives other sounds as separate - If a computer could do that it could cleanly take a sample of just the drums with everything else removed - up to now that hasn't been possible)

I don't know if it's such a great thing -

In the mid to late 80s - recordings were being re-mastered using small memory samplers to tweak and 'fix in the mix' little bum notes here and there.

Melodyne looks impressive - but I don't think it's the Alpha and Omega

(saying that I'd love to have it and a machine powerful enough to do the processing to run it)

Actually though - I think it probably should just be used for slightly tweaking small bits of badly out of tune performance - not hammering everything.

(but there is that other cool thing it can do of fixing notes that are out of time from say a guitar - but I've only seen that in the demo vid - don't know if really does it)

studiorat

I really don't think timbre has anything to do with it. It would be much more difficult to detect a difference in timbre on the strings of the acoustic guitar in the demo than pitch differences between the strings.

The amount of timbre information that can be extracted is still very small, a lot of this technology is currently being used for meta-data for mp3 type files, to identify content in them to try and catageorise them.

As far as I can gather Melodyne works along the following principles...

That, 1 most musical instruments have a strong base frequency (fundamental), and harmonics with an odd or even multiple.

2, you do a fine FFT and "look" from the lowest frequency upwards, the first strong lowest will be the lowest note in the chord.

3, you multiply by 2,3,4,...n and you find all the harmonics related to that lowest note.

4, move on from this first note until you hit the next strong frequency which is not a multiply of the first tone, and so on...

It would make a great tool for synthesis. I'd really like to have a go of that JJ, what's the story with getting a look at the beta?

kfoltman

studiorat wrote: »

2, you do a fine FFT and "look" from the lowest frequency upwards, the first strong lowest will be the lowest note in the chord.

Not necessarily. The fundamental and first few harmonics may not be present at all The signal doesn't even need to be particularly "pathological", highpass filtering is enough to confuse this method while still allowing the human ear to perceive the pitch

There are some other approaches to pitch estimation (some may be based on FFT, others may not - example: autocorrelation). This is all rather tricky anyway, especially when applied to signals that are polyphonic, full of transients, or both.

This kind of sound analysis and processing was previously available as research-oriented software only (example: open source Sonic Visualiser). Nice to hear that it's finally available in form that's acceptable for wider audience!

studiorat

It would seem autocorrelation is used to extract the pitch in Auto-Tune...

krd

It could be something like this

http://www.guardian.co.uk/science/blog/video/2009/sep/07/music-analysis-dan-stowell

if you see how the timber can be separated.

---

From the melodyne demo I've seen - it seems melodyne aims to isolate actual "notes" not just the frequency.

But I have a feeling it's a combination of several different tricks.