Nov 06, 2007
FStein -
Hello and welcome to my blog! My name is Dr. Franklin Stein, new director of Columbia's School of Neuroscience specializing in modeling and synthetic neural nets. I'm very excited about the work we have planned this semester at Columbia, some very innovative new technologies I'll be sharing with you over the web. And as part of Columbia's new OpenLearning initiative, I'll be sharing many of the images, models, presentations and research papers here in an online format!
Before I get over my head though, I'd like to share a little bit of background on the field of synthetic neuroscience, specifically what we're doing here at Columbia. It's a very cutting-edge field. It's not often you get to do things that have never been done before, especially in the hard sciences.
Neuroscience is still a very young field. It was only 100 years ago that phrenology was in fashion - the belief that the shape of the skull determined the workings of the brain! It sounds preposterous now, but that's because we've come so far in such a short time. Now we know that the brain is made up by trillions of long spindly cells called neurons. Each neuron is sort of like an electrical switch, and it can either be turned on or off. Synthetic neurons have been created in the lab, even implanted in rats and other mammals!
More importantly, though, are the connections and networks created by the neurons. Each neuron has hundreds or thousands of little tails, called dendrites. You can almost think of them as little arms (see image 1), reaching out and connecting with other neurons. Even a handful of neurons can have hundreds of connections with each other. You can see how quickly everything can add up into an incredibly complex network!
That's where the modern field of neurology comes in. The goal of neurology research is to isolate networks of neurons that do specific jobs. For example, neurologists have determined the set of neurons responsible for seeing straight lines. So every time the signal from the eye travels to the brain, this group of neurons (roughly 200,000) fires off and identifies a straight line. Even more interesting is the same network exists in rats, cats, monkeys and humans!
Here in synthetic neurology we take the goal a step further. We construct networks of neurons to perform jobs.
Three years ago some colleagues of mine managed to train a set of rat neurons to fly a plane. Incredible, right? That's the amazing thing about these neural networks - they can be trained.
Every time a neuron fires - that is, sends it signal out to its neighbors via its dendrites - its chemical makeup is changed. The chemicals inside the cell determine how easy it will fire - it's potential. The more it fires, the lower the threshold, and therefore the higher potential to fire again. That's why addiction grows and tolerance goes up - the brain is adjusting to increasing stimulus.
In the case of the rat pilot, the scientist trained the neurons to seek equilibrium - where the plane was flying smoothly. If turbulence or strong side wind hit the wings, the neurons would adjust until the plane leveled out again. They even tested it successfully in hurricane gale winds!
We've come a long way since those first experiments with rat neurons. Here at the Columbia Neuroscience laboratory, I'm pleased to announce we'll be doing some ground breaking studies with entire-map human neural net modeling. In conjunction with the Columbia School of Medicine, Columbia School of Computer Science, IBM Supercomputer group, and the New York Department of Corrections, we'll be modeling the entire human brain on computers!
Thanks to the New York Department of Corrections, we've managed to obtain a number of specimens who have donated their bodies to science. We're going to put them to good work constructing the first working model of the human brain in software.
Stay tuned for tomorrow's blog post when I'll introduce my team, and go into some more detail on how we're going to approach this amazing new opportunity.
***
Nov 8, 2007
FStein -
Welcome back! Before I get started today, I want to encourage you to check out the new uploads in the video section. Together with Igor Akwakanan, we received the first shipment of specimens from Riker's Island, one of New York's biggest correctional facilities. We really got our hands dirty prepping the craniums, but its all in good fun. Nothing wakes me up in the morning like a solid session on the bone saw!
As I mentioned, Igor is my assistant (see image 2). He's a fourth year doctoral student in computational neurology, and he's already completed studies in medical forensics. Comes in handy with specimen work, especially the anatomy of the cranium and brainstem.
In addition to Igor, there's Xing Exiwuan and his team, working in conjunction with IBM consultants on the hardware side, Petyr Oliverv from the school of Computer Science in charge of software and data flow technologies; and Renahldo Baas assisting on the brain scanning instruments. Finally, there's Marr Shelby who's going to be helping me document this entire adventure.
If you get the chance, check out the photo gallery. It includes our entire team out on Halloween. Most of us went in our lab coats, but Igor really blew us away with his interpretation of classic film star Boris Karloff.
Ok! Now that you've met the team, I'm sure you're very interested in finding out how we'll be constructing the model of the human brain. Theoretically, it's very simple. We're taking very thin slices of tissue, snapping a photograph and then feeding that information into the computer database. We've only recently developed the technology with the precision required.
The first gadget is the TEAR - Tissue Extraction and Amplification Rotor. It's basically a super expensive meat slicer like those used in a deli. However, instead of slices a few millimeters thick, these slivers are only one micrometer thick. Of course, tissue that thin just disintegrates into a pile of pink mush, so we take the picture before we do the slicing, and then build the model of neurons based on what's removed. Using a high powered thermal laser, we skim off a thin layer at a time.
The next important device combines two cutting edge technologies in a single 20 million dollar box. The Magnification Amplification Reflection Reactor (MARR) feeds into the Rotating Expansion and Notation Device (REND) to capture the high resolution image, identify the neurons and dendrites, translate this information into relevant database fields. Each 50 terabyte image is then stored in our data center for future processing and annotation.
Beyond that - we put the brain back together in software and run it as fast as we can on IBM supercomputers. It will be difficult to have the thing run real-time. Our best hope is one-tenth of human processing speed, and it could be much slower if the organization of the neural network in software isn't optimized. Over time, however, we should be able cut down on extraneous calculations and see some real results.
I've got my work cut out for me (literally!), so it'll be a few weeks before I can update. Hopefully the next time you hear from me, we'll have scanned in an entire brain.
***
Nov 20, 2007
FStein -
Success! I'm happy to say that the last two weeks have been some of the most exciting of my career. Twenty-hour days and a little bit of luck has paid off. We have a functional neural net modeled in software!
I couldn't have done it without my team, Igor especially, who kept the entire operation going, fueled primarily on Starbucks and Big John's pizza. Granted, there were some trying times, especially during the first weekend when we miscalibrated the TEAR and sliced the cranium specimen in half. We lost about 900 tb of data and had to start over from scratch. But the team just dug in their heels, locked the skull in tight and got to work.
I must admit, it got a little crazy back there when the coagulated brain matter started to gum up the camera, misting the lens with pink residue. Igor developed a method for switching out the lens in under five minutes and we ran dehumidifiers at full blast to cut down on the air moisture. (Check out the picture's Marr has posted. Image 1 - slicing in progress. Image 2 - bisected skull, yuck!)
But enough about the dirty trials and tribulation, what about the success? The outcome. I'm sure that's what you're all wondering.
What we have running at this moment is a massively parallel simulation of a brain on an IBM supercomputer. It was only just yesterday I remember Igor running into my office, screaming "It's alive!"
I was excited along with him, but I had to quickly correct him. This is only a simulation. We can pause and even rewind the sequence of neural chain reactions. I'd even go so far as to say the simulation lacks any sort of consciousness. Just think, it contains storage of memories, patterns of speech, mannerism, random facts. But none of these are tied into a physical body.
For the most part, we've left the simulation "beheaded". What I mean by that is we haven't connected the muscle and nerve fiber neurons up to any sort of feedback loop. All the lower level brain stem functionality is disabled. We're only interested in the upper level processing facilities - memory, language processing, logic, higher-level rationalizations.
It took us some time, however, to figure out how to "test" all of this. This is a brain in a jar without wires to the outside world. It's trapped in a dark vacuum.
But Nimesh Kadawan, one of our computer science graduate students, came up with the bright idea of immersing our sim inside a virtual world. Nimesh figured the best representation that would trigger a reaction would be simple enough - an 8 x 8 cell. Our specimen was familiar with the grey and green facilities on Riker's Island. After all, he spent ten years on death row there.
With a bit of fiddling, we managed to connect the rendered pixels of the virtual world into the visual neurons. And while Nimesh controlled the actual movement with his keyboard and mouse, everything displayed on the screen would be "playing" inside our sim's mind. (image 3 - rendering of cell block A, complete with cot, toilet, book, plastic shiv)
What we saw amazed us. Entire neuron chains were lighting up like fireworks. There was activity in the memory banks of the temporal lobe, reasoning in the prefrontal cortex, emotions in the hypothalamus, and of course "sight" in the visual lobs.
Two all-nighters and Nimesh programmed in hearing, using a similar methodology to mimic the ocular nerves. We could now speak to our man. We played a tape of common prison sounds - yelling, discipline, beatings, chow hall. All fired off signs of recognition, mental processing. We were making amazing progress.
But the big question was how to get any information back out. We could see neural nets firing in the frontal cortex - a sign of some sort of recognition. But we had no idea what any of it meant. He might be thinking "where the hell am I?" or "I can't feel my legs." But to us it simply looked like a bloom of dots on the monitor. Deciphering exactly what all those incredibly complex networks meant is beyond current science.
What we needed to construct was some way for our Sim to interact with the virtual world. We needed to train him.
Again, this is not new stuff. It's been done with paraplegics in hundreds of documented cases. Handicapped victims who have lost the use of limbs have learned how to operate robotic arms or wheelchairs with their minds. All it takes is a little training, and the synching of the robotic tool with an actual set of neurons.
In our case, we needed to teach our Sim how to move around his cell and interact with objects in the world. Nimesh coded up a stack of big plastic alphabet letters and tossed them in the cell. Then we began some rote exercises.
Nimesh would move forward in the cell and we'd manually activate a set of neurons correlating to the leg muscles. Then we displayed a blackboard in the cell, written with the words "This is how you move forward." After a half dozen tries, the sim could move on it's own.
We did the same thing with the blocks, educating the Sim on how to pick up an object, rotate it, set it down again. The frustrating thing is the current optimizations allow only one fiftieth of actual human processing, so it feels like we're teaching a very mentally disabled child. That's the price we pay for progress!
Tonight, we've left the Sim in its cell to see if it spells anything out in the letters. Tomorrow, we shall see what we shall see...
* * *
Nov 21, 2007
FStein -
(Image 1 - Alphabet block letters, spell roughly "im bob")
We have a name! The Sim has introduced himself as Bob.
The team was ecstatic when we arrived in the morning to see the letters placed on the bed, if a bit messy, at least decipherable. I immediately picked up the mic and spoke into his mind. "It's nice to meet you Bob. How are you feeling?"
We then had to wait for another half hour, staring at the tiny rendering of the virtual world as Bob jerked around, picked up letters and rearranged them.
"is this hell" he spelled out. There was nervous chuckling and I think Nimesh looked a little pale. I decided to pause the simulation while we decided our next course of action.
We could always rewind the various simulations we had taken, but processing power was at a premium. For every bad choice that drove Bob to insanity or despair or broke the illusion of reality, it could take days if not weeks to erase what we'd done, especially if the seeds of doubt were sown earlier. I decided it would be better to advance one solid step at a time, with frequent database backups, introduce Bob to his new world slowly.
Nimesh and some of the IBM guys advocated spelling everything out. "Give him the red pill," they said, an allusion to some science fiction movie (can anyone answer in the comments?). But I made the executive decision. I would be the voice inside Bob's head - God, if you will - guiding him into his new world.
"Bob," I said, after we resumed the simulation. "You have been given a second chance. All your sins have been forgiven." I took a gamble, given the hell reference. "Your only responsibility is to explore your world, to learn."
Nimesh wanted to introduce humanoid models into the virtual world, but I was reluctant despite his best efforts. Even the high polygon models had a strange zombie look, and I didn't want to scare off Bob. We decided to open up the cell doors into a field of trees, flowers, butterflies, sunsets - one of the better off-the-shelf 3D screensavers.
Nimesh also managed to code a pen and a large roll of paper. "Bob, I'm going to teach you how to write." We picked a quadrant of neurons near his hand motor nerves, lining out a grid to correspond to motions of the pen on paper, activated when he was in "writing mode."
"Here's what it feels like with the pen in your hand," I said, stimulating the middle of the quadrant. "Now write up," I said, firing the north quarter. A vertical line appeared on the white field of paper. "Down," I said, hitting the opposite side. "Left, right. Good Bob. Let's try a circle."
Soon we were on to letters of the alphabet, slowly tracing through the curves and jots of A, B, C. When we got to E it was a bit tougher, requiring a bit more concentration to trace through the bars.
"It'll be some time before he can write with any speed," I said to the group. "Right now he has to utilize a good portion of his frontal cortex to "think" about moving his hand. It'll take some time before he develops "muscle memory" and doesn't have to concentrate on each motion."
What happened next surprised even me. Instead of continuing through the alphabet, Bob began to scribble in large furious strokes, letters eventually taking shape when we zoomed out on the screen resolution. F, U, C, *, Y, O, U...and I'm sure you can fill in the rest. Bob was not happy. Not good.
Igor ran out and got us coffees and I wondered whether to freeze the sim. Nimesh convinced me to leave it running, to let Bob cool off in the field of blowing grass and rendered sunsets. In his past life, he was a convicted felon after all, sentenced to death row. That hatred was somewhere hardcoded in his neurons.
But I have faith in the plasticity of the brain. If Bob's brain can learn to write with a pen inside a virtual world hardwired into his visual cortex, he can learn stress management.
More updates tomorrow...
* * *
Nov 23, 2007
FStein -
I apologize for not posting. Things have been extremely hectic lately. And unfortunately, not all of it has been positive. We had to let some team members go this past week due to a breach of Columbia's Academic Honor Code.
Since it's an ongoing investigation, I won't list names, but I will say that the consequences of their actions may have compromised everything we've accomplished with Bob. For the time being, I've had to suspend the entire simulation while we recover what we can.
First, a little background. As I mentioned in my previous post, there was a significant faction of researchers who believed it too cruel to confine Bob to our private virtual world. Unbeknownst to me, and without my permission, they introduced Bob to a public server virtual world, loose on the internet. Later, they explained that Bob needed contact with human "avatars," not simply voices in his head and magically appearing alphabet messages.
This is incredibly frustrating. Bob was loose in an uncontrolled environment for over ten hours, and while his processing was still a bit slow - maybe as little as fifteen minutes of real-time human thinking - it was long enough to expose him to unmonitored communication with non-professionals. I have no idea what he said to them - the chat logs were deleted.
Furthermore, the same malicious group of grad students corrupted the database logs, preventing a roll back or rewind of Bob's thought processes. There's only what we have now, the sequence of neural connections that have been tarnished by the World of Warcraft!
(image 1 - large minotaur creature holding an Axe in a cartoonish forest)
Next steps (aside from drinking copious amounts of vodka) are to reintroduce Bob to the sterile environment - his cell block. It will take some time to build up a knowledge base in the team - the inner workings of the visual and audio interfaces are still a bit cryptic even to me. Igor says pieces of code were even deliberately obfuscated.
The most frustrating thing about this - why would such a promising student throw it all away? These individuals were shooting stars, on the path for amazing careers.
One in particular had given up so much for the project. I think this past month alone he worked five hundred hours, unceasingly, thousands of lines of code, hundreds of recompiles, sleeping and eating at his desk. And yet he was so stoic when I told him to clean out his desk, turn in his building keyfobs. He looked straight at me and said I was "torturing a soul."
I'm going to be taking a break from the blog, reconnect with Bob, see if we can unscramble the mess in the software. But all is not bleak or lost. Think of the amazing success we've had. Whether we've brought a mind to life or merely modeled an incredibly accurate representation remains to be seen. Either way, it's incredible, and I truly am a privileged man.
Remember, seek out knowledge and you will find wisdom. I ardently hope that the gratification of your wishes may not be a serpent to sting you, as mine has been.
* * *
December 10, 2007
Mshelby -
I regret to inform our readers that the OpenLearning portion of Dr. Stein's research has been discontinued, pending an investigation into his death this past Sunday. This blog will remain online but all hosted images, slides, documents, models and videos will be removed at 12:00 AM Dec 11th. At this time, the project is suspended indefinitely.
Thank you for being part of history.
Below is a link to Dr. Stein's obituary in the New York Times:
Dr. Franklin N. Stein, a leading research scientist at Columbia University prestigious School of Neuroscience, was found dead in his Upper West Side brownstone Apartment Nov 27th. He was 63. New York Police Department responded to reports of a struggle from his neighbors. Foul play is suspected.
"This is an ongoing investigation," said Det. David Holmes. "In addition to multiple stab wounds, there were a number of large plastic letters stapled to his chest, spelling out FOR BOB."
Columbia University said that there had been a number of departmental issues recently. Igor Akwakanan, Stein's assistant, tearfully recounted how Stein was working on his brain child, and expressed hope of a Nobel Prize in medicine. "It's all over now. Bob's gone."
|