Scientists Reconstruct Video Clips From Brain Activity

TPD

UC Berkeley scientists have developed a system to capture visual activity in their brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.

I just can't believe this is happening for real, but according to Professor Jack Gallant—UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology—"this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds."

Indeed, this is mindblowing. I'm simultaneously excited and terrified. This is how it works:

They used three different subjects for the experiments (incidentally, they were part of the research team because it requires to be inside a functional Magnetic Resonance Imaging system for hours at time and nobody wanted that job). Inside the machine, they were exposed to two different groups of Hollywood movie trailers as the fMRI system recorded the brain's blood flow through their brains' visual cortex.

The readings were fed into a computer program, in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain activity. As the sessions progressed, the computer kept learning about how the visual patterns presented on the screen corresponded to the brain activity.

After recording this information, the activity from the second group of clips was used to reconstruct the videos shown to the subjects on a computer screen. The computer analyzed 18 million seconds of random YouTube video, building a database of potential brain activity for each clip. From all these videos, the software picked the one hundred clips that looked more similar to the ones the subject watched, combining them into the final movie. Although the movie is low res and blurry, it clearly matched the actual clips watched by the subjects.

In this other video you can see how this process worked in the three people. On the top left you can see the movie the subjects were watching while they were in the fMRI machine. Right below you can see the movie "extracted" from their brain activity, which shows that this technique gives consistent results independently of the human or content. The three lines of clips next to the left column show the random movies that the computer program used to reconstruct the visual information.

Right now, it's low res and imprecise, but the potential is enormous. Lead research author and one of the lab test bunnies Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines:

Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences.

That will include your visual memories and dreams.

This is the first time in history that we have been able to decode brain activity and reconstruct motion pictures in a computer screen. The path that this research opens boggles the mind. It reminds me of Brainstorm, the cult movie in which a group of scientists lead by Christopher Walken develops a machine capable of recording the five senses of a human being and then play them back into the brain itself.

This new development brings us closer to that goal which, I have no doubt, will happen at one point. Given the exponential increase in computing power and our understanding of human biology, I think this will arrive sooner than most mortals expect. Perhaps one day you would be able to go to sleep with a Sony Dreamcam connected wrapped around your head like a beanie.

That's some futuristic **** right there.

Woops, a link: http://gizmodo.com/5843117/scientists-reconstruct-video-clips-from-brain-activity

Saila

if I wanted my inner most thoughts broadcast I would say them out loud!

stop science, just stop!

going by the videos there that really is impressive stuff! :eek:
I can see how the images are made and how the association of one image can trigger another, especially the plane and the city scape, pre 911 that image would have been very different I would say

LH Pathe

Sounds creepy as fück. What if it got to the point it got the go ahead to put in CCTV, prying on our daydreams us ambling along

pdfile would be like hey its the thought police maaaan, cramping my style

First the potentially faster than the speed of light thing and then this.. Two.of.the most impressive steps I've heard in a while.

pragmatic1

Pretty massive implications.

texidub

The Singularity cometh....

danniemcq

ohhh i hope neither my family or GF see what i dream of....


so much blood

stovelid

Sometimes science just goes too far.

KilOit

Should be using porntube videos instead if they wanted to get a database of dreams

LumpyGravy

In the clip why is the brain seeing a dude in a T-shirt when the presented clip shows Steve Martin dressed as Clouseau?

mikom

In a cost cutting measure RTE are to broadcast Ryan Tubridys dreams for 3 hours every Friday night.
Expect to see JFK dressed as Sorcha Ronan whilst snorting jelly beans from a box set of Mad Men to the strains of Frank Sinatra.

Handy11

They did it in House and it was class. Obviously dramatised/fictionalised, but the idea is fantastic.

JerryHandbag

This reminds me, Fringe is back on telly soon

TheMilkyPirate

So after reading through the comments, I thought of something: What if the technology becomes so advanced you can't tell it from any other video? We could essentially dream up anything and have it on film. This would totally destroy the film CGI industry... instead of having an army of technical artists, you would only need a few really imaginative people. Scary.

:eek:

MagicSean

So long Hollywood. My dreams are much more awesome than any movie.

Handy11

But I've neva been to maawss.

mikom

Handy11 wrote: »

But I've neva been to maawss.

If I'm nart me, who dah hell am ah.

Handy11

mikom wrote: »

If I'm nart me, who dah hell am ah.

PMSL at this perfect piece of phonetics

--Kaiser--

stovelid wrote: »

Sometimes science just goes too far.

You say 'sometimes science goes too far,' I say sometimes it doesn't go too far enough!

Logical Fallacy

That is really cool but their actual science infringes on a screenplay that i am writing.

Bastards.

biko

https://www.youtube.com/watch?v=s0zaqWQiXG8

ejmaztec

mikom wrote: »

In a cost cutting measure RTE are to broadcast Ryan Tubridys dreams for 3 hours every Friday night.
Expect to see JFK dressed as Sorcha Ronan whilst snorting jelly beans from a box set of Mad Men to the strains of Frank Sinatra.

Not another feckin repeat:mad:

GetWithIt

Seanbeag1 wrote: »

So long Hollywood. My dreams are much more awesome than any movie.

So long internet pron. My dreams. Your Ma. So hot.

Korvanica

Awesome, next step..dream recording pillows

TheIrishGrover

Sounds like Wim Wenders' 1991 movie "Until The End Of The World". In it they had a pair of glasses that could record video as well monitoring brainwave patterns while the user was recording. Then they were able to stimulate these same patters into a blind person's brain to replicate the viewing experience. However then they started using it to record their brainwave patterns while sleeping and replicate them again when awake to see experience the dream while awake.

Orando Broom

mikom wrote: »

If I'm nart me, who dah hell am ah.

Shtort the reachtor Quuhaid.

Seachmall

Imagine what this would mean for the courts. Everybody would essentially be a walking security camera (although entirely an fallible one) and witness statements could become witness reconstructions.

Also, I could rewatch sex dreams.

3ndahalfof6

big difference between what your thinking and what you are imaging inside the oul head

orourkeda

At least my dreams will be banned on youtube