tangent4ronpaul
05-30-2013, 02:49 AM
http://www.pbs.org/newshour/bb/science/jan-june13/facial_05-29.html
Transcript
GWEN IFILL: Now to the second of our stories on the role of technology in unraveling the Boston bombing case.
Last night, science correspondent Miles O'Brien traveled to an explosives testing facility to learn more about the bomb itself. Tonight, as part of his work for a special NOVA program, he reports on the facial recognition software that allowed investigators to identify the bombing suspects.
MILES O'BRIEN: It is a small unit on the cusp of a big change in the way police do their job. Welcome to the New York Police Department's facial identification section.
Inspector Ken Mekeel is the man in charge.
Do you feel like you're sort of at the beginning of when fingerprints first came in? Is this the beginning of where policing is headed, in many respects?
INSPECTOR KENNETH MEKEEL, New York Police Department: Yes, very much so. It's not as good as a fingerprint.
MILES O'BRIEN: Right.
KENNETH MEKEEL: This doesn't have the confidence level of that. It's not a definitive science such as fingerprints and DNA. But this is good.
Just basically, maybe we can get a lead. Maybe we can be pointed in a direction that this might be a possible person. Hollywood makes facial recognition look easy. But when you face the facts, this law enforcement tool is not as easy as it looks in the movies.
Mekeel gave us a demonstration, a scenario similar to the Boston Marathon bombings: a grainy surveillance image of a suspected terrorist. First, they look for the best picture.
KENNETH MEKEEL: This image, as you can see, has the head a little turned. The distance is a problem. This particular head is down. There's really not much face. And we are looking for -- comparison and facial recognition software is a frontal pose.
This seems to be the best picture that we would utilize in this type of situation, so we would grab that picture, basically cut it out.
MILES O'BRIEN: But the software only works if the person is squarely facing the camera. And this angle is not good enough. But there is a way to fix that. They map out the suspects face, placing crosshairs on several key features: ears, nose, eyes, chin, and mouth.
Once the software knows where all those features are, it uses a formula to convert the slightly askew photo into a three-dimensional image. And that allows them to turn the suspect's face directly toward the camera virtually.
KENNETH MEKEEL: So, now we're using the frontal pose. It's called the normalization pose. We're going to enter it into the facial recognition software. So, we want to use this probe image from an unknown individual against our gallery of known individuals. And we basically have close to four million images that we use within the NYPD.
MILES O'BRIEN: The images are mug shots, but in this demonstration, they are actually cops. Police here say they don't look for a match by trolling through driver's licenses, passport or visa photos, much less Facebook.
KENNETH MEKEEL: So, now, in order to put that in, we input it into our facial recognition software. We want to line up the eyes as best as possible, try to center it. And you can see, even though this picture is not aesthetically nice to humans looking at it, you can see that it can work, though, within the software.
MILES O'BRIEN: In the demo, the face is a match. But this makes it look a littler easier than it is. The database is relatively small. And, in this case, the perp is Detective Roger Rodriguez, assigned to the unit, and sitting in the back row watching the boss do the demo.
KENNETH MEKEEL: The problem that we have within law enforcement doing criminal investigations with facial is that we are not utilizing controlled images as probes, so the unidentified person is a bad image for the most part.
MILES O'BRIEN: So far, this unit has analyzed 1,900 images, turned up more than 386 matches, and that has led to 141 arrests.
KENNETH MEKEEL: The optimum scenario is a good high-resolution picture, very well-pixelated. We need it to the point where the lighting is correct, the distance is not too far, and those are easier to manage and to work with within the facial recognition system.
MILES O'BRIEN: In Pittsburgh, at Carnegie Mellon University's Biometrics Center, engineer Marios Savvides is leading a team that is pushing facial recognition to the next level.
When he saw the photos of the Boston Marathon bombing suspects released three days after the attack, he sprang into action, even though the odds were slim that he could do anything to help investigators in Boston.
MARIOS SAVVIDES, Carnegie Mellon University Biometrics Center: The problem in the Boston case is that the image was so small that technology cannot handle that low-resolution face. There's simply not enough information. So you have to employ super-resolution techniques and image-enhancement techniques to basically, what we call, the computer has to hallucinate.
And I hate to use that word, but if you look at the input image and you look what the computer or algorithm can develop, you will almost think, well, how on earth it was able to actually extract that?
MILES O'BRIEN: How, indeed? The best possible image was this one, a very low resolution, poorly lit, but nearly full-frontal image of the younger Tsarnaev brother, Dzhokhar. Savvides and his team plotted 79 distinct points on his face as best they could.
MARIOS SAVVIDES: We will click on the image, and this is what you're really seeing. And you can see you really don't see a lot of facial structure.
MILES O'BRIEN: Right.
MARIOS SAVVIDES: So we had to guesstimate, estimate where those landmarks are on the face.
MILES O'BRIEN: They fed the blurry image with the markings into their latest facial recognition algorithm.
MARIOS SAVVIDES: Our algorithm was able to basically enhance and extract this face. So this is the face that came from that.
MILES O'BRIEN: Savvides calls it a super-resolution enhanced image. He intended to send it to the FBI, but by the time they finished, it was 2:42 Friday morning. The shoot-out in Watertown was over and the brothers were identified.
Now there were plenty of high-resolution shots to choose from. On the surface, it seemed as if Savvides' computer hallucination wasn't very accurate or useful. But computers see the world differently. So Savvides tried an experiment. He added a high-resolution frontal image of Dzhokhar Tsarnaev to a database of one million images.
MARIOS SAVVIDES: So, we want to see, could we find him? Is he in the top 100? Where is he? Did what we enhance actually be able to match one of the Tsarnaev's frontal images?
MILES O'BRIEN: When compared to the computer-generated face, the Tsarnaev image ranked 1,417.
But when Savvides narrows the search, it gets more interesting. After ruling out women, non-Caucasians, heavy facial hair and those over 25, the photo ranks number 20. For Marios Savvides, this was a eureka moment.
MARIOS SAVVIDES: Based on the results that we have, we see that they're accurate. This is -- our work is at an infancy, but we're still blown away on how it works. I mean, we still can't believe how amazing we got. When we enhanced that image and we go -- and I saw, well, this wasn't a random face.
MILES O'BRIEN: All right, smile or just neutral?
So how does it work? I volunteered my face for a demonstration. The reduced-resolution image, 25 pixels across the eyes, leads to this interpretation.
MARIOS SAVVIDES: But the general face structure, that is pretty much well-maintained. All these features are there from this.
MILES O'BRIEN: What about 12 pixels between the eyes, the kind of image you would typically get from a CCTV camera?
MARIOS SAVVIDES: So let's see what happens when we run our algorithm. OK. Wow.
MILES O'BRIEN: Not bad.
MARIOS SAVVIDES: Not bad. So we went from this image to this image.
MILES O'BRIEN: Yes.
MARIOS SAVVIDES: And if we compare it to the actual high resolution, again, the eyebrow structure is there, the nose, the eyes, the mouth. We have lost the high-frequency texture because there is no information. But the structure of the face is there.
MILES O'BRIEN: How low can it go? What about six pixels from eye to eye?
MARIOS SAVVIDES: So from this to get this, that's amazing, and that's how computers can be better than human minds, because I could not extrapolate this information from this.
MILES O'BRIEN: Boy, I should have shaved, huh?
Anyway, you're really getting every whisker.
The secret is machine learning and pattern recognition. Instead of trying to tell the computer how to turn a few pixels into a recognizable face, Savvides simply shows it thousands and eventually millions of examples of the same faces in very low and very high resolution. Over time, the computer identifies patterns and makes connections that humans cannot. It learns.
MARIOS SAVVIDES: That's where computers can be smart, and that's where it can do things that our human brain cannot do, because it can learn the relationship of what a low-resolution, degraded-face image is, and what the high res. When you give it enough data to learn that relationship, then it can actually build an algorithm to extract the high-res face that you can feed into such systems to try to give you a match.
Well, if you use the periocular, we're not looking at the mouth.
MILES O'BRIEN: So, no expression?
MARIOS SAVVIDES: It doesn't affect it all.
MILES O'BRIEN: Expression can be a problem for you, right?
This is not all they are working on here. They are looking for ways to identify faces simply by focusing on the eyes or even just the eyebrows. They are predicting age better than a carnival barker. They are working on putting facial recognition software into smartphones, and they are developing sophisticated infrared cameras that scan irises.
COMPUTER VOICE: Identified. Welcome, Miles.
MARIOS SAVVIDES: When law enforcement has nothing to work on, have no leads, any lead is something. So there's a chance it may be wrong. It was wrong, OK, but it was a lead. But if it was right, if it was able to get you an image in that top 50, then that's huge. And this is the thing. How can you give the investigators something to work on?
MILES O'BRIEN: Facial recognition may not measure up to the movies just yet, but it is coming soon to a police department near you.
-t
Transcript
GWEN IFILL: Now to the second of our stories on the role of technology in unraveling the Boston bombing case.
Last night, science correspondent Miles O'Brien traveled to an explosives testing facility to learn more about the bomb itself. Tonight, as part of his work for a special NOVA program, he reports on the facial recognition software that allowed investigators to identify the bombing suspects.
MILES O'BRIEN: It is a small unit on the cusp of a big change in the way police do their job. Welcome to the New York Police Department's facial identification section.
Inspector Ken Mekeel is the man in charge.
Do you feel like you're sort of at the beginning of when fingerprints first came in? Is this the beginning of where policing is headed, in many respects?
INSPECTOR KENNETH MEKEEL, New York Police Department: Yes, very much so. It's not as good as a fingerprint.
MILES O'BRIEN: Right.
KENNETH MEKEEL: This doesn't have the confidence level of that. It's not a definitive science such as fingerprints and DNA. But this is good.
Just basically, maybe we can get a lead. Maybe we can be pointed in a direction that this might be a possible person. Hollywood makes facial recognition look easy. But when you face the facts, this law enforcement tool is not as easy as it looks in the movies.
Mekeel gave us a demonstration, a scenario similar to the Boston Marathon bombings: a grainy surveillance image of a suspected terrorist. First, they look for the best picture.
KENNETH MEKEEL: This image, as you can see, has the head a little turned. The distance is a problem. This particular head is down. There's really not much face. And we are looking for -- comparison and facial recognition software is a frontal pose.
This seems to be the best picture that we would utilize in this type of situation, so we would grab that picture, basically cut it out.
MILES O'BRIEN: But the software only works if the person is squarely facing the camera. And this angle is not good enough. But there is a way to fix that. They map out the suspects face, placing crosshairs on several key features: ears, nose, eyes, chin, and mouth.
Once the software knows where all those features are, it uses a formula to convert the slightly askew photo into a three-dimensional image. And that allows them to turn the suspect's face directly toward the camera virtually.
KENNETH MEKEEL: So, now we're using the frontal pose. It's called the normalization pose. We're going to enter it into the facial recognition software. So, we want to use this probe image from an unknown individual against our gallery of known individuals. And we basically have close to four million images that we use within the NYPD.
MILES O'BRIEN: The images are mug shots, but in this demonstration, they are actually cops. Police here say they don't look for a match by trolling through driver's licenses, passport or visa photos, much less Facebook.
KENNETH MEKEEL: So, now, in order to put that in, we input it into our facial recognition software. We want to line up the eyes as best as possible, try to center it. And you can see, even though this picture is not aesthetically nice to humans looking at it, you can see that it can work, though, within the software.
MILES O'BRIEN: In the demo, the face is a match. But this makes it look a littler easier than it is. The database is relatively small. And, in this case, the perp is Detective Roger Rodriguez, assigned to the unit, and sitting in the back row watching the boss do the demo.
KENNETH MEKEEL: The problem that we have within law enforcement doing criminal investigations with facial is that we are not utilizing controlled images as probes, so the unidentified person is a bad image for the most part.
MILES O'BRIEN: So far, this unit has analyzed 1,900 images, turned up more than 386 matches, and that has led to 141 arrests.
KENNETH MEKEEL: The optimum scenario is a good high-resolution picture, very well-pixelated. We need it to the point where the lighting is correct, the distance is not too far, and those are easier to manage and to work with within the facial recognition system.
MILES O'BRIEN: In Pittsburgh, at Carnegie Mellon University's Biometrics Center, engineer Marios Savvides is leading a team that is pushing facial recognition to the next level.
When he saw the photos of the Boston Marathon bombing suspects released three days after the attack, he sprang into action, even though the odds were slim that he could do anything to help investigators in Boston.
MARIOS SAVVIDES, Carnegie Mellon University Biometrics Center: The problem in the Boston case is that the image was so small that technology cannot handle that low-resolution face. There's simply not enough information. So you have to employ super-resolution techniques and image-enhancement techniques to basically, what we call, the computer has to hallucinate.
And I hate to use that word, but if you look at the input image and you look what the computer or algorithm can develop, you will almost think, well, how on earth it was able to actually extract that?
MILES O'BRIEN: How, indeed? The best possible image was this one, a very low resolution, poorly lit, but nearly full-frontal image of the younger Tsarnaev brother, Dzhokhar. Savvides and his team plotted 79 distinct points on his face as best they could.
MARIOS SAVVIDES: We will click on the image, and this is what you're really seeing. And you can see you really don't see a lot of facial structure.
MILES O'BRIEN: Right.
MARIOS SAVVIDES: So we had to guesstimate, estimate where those landmarks are on the face.
MILES O'BRIEN: They fed the blurry image with the markings into their latest facial recognition algorithm.
MARIOS SAVVIDES: Our algorithm was able to basically enhance and extract this face. So this is the face that came from that.
MILES O'BRIEN: Savvides calls it a super-resolution enhanced image. He intended to send it to the FBI, but by the time they finished, it was 2:42 Friday morning. The shoot-out in Watertown was over and the brothers were identified.
Now there were plenty of high-resolution shots to choose from. On the surface, it seemed as if Savvides' computer hallucination wasn't very accurate or useful. But computers see the world differently. So Savvides tried an experiment. He added a high-resolution frontal image of Dzhokhar Tsarnaev to a database of one million images.
MARIOS SAVVIDES: So, we want to see, could we find him? Is he in the top 100? Where is he? Did what we enhance actually be able to match one of the Tsarnaev's frontal images?
MILES O'BRIEN: When compared to the computer-generated face, the Tsarnaev image ranked 1,417.
But when Savvides narrows the search, it gets more interesting. After ruling out women, non-Caucasians, heavy facial hair and those over 25, the photo ranks number 20. For Marios Savvides, this was a eureka moment.
MARIOS SAVVIDES: Based on the results that we have, we see that they're accurate. This is -- our work is at an infancy, but we're still blown away on how it works. I mean, we still can't believe how amazing we got. When we enhanced that image and we go -- and I saw, well, this wasn't a random face.
MILES O'BRIEN: All right, smile or just neutral?
So how does it work? I volunteered my face for a demonstration. The reduced-resolution image, 25 pixels across the eyes, leads to this interpretation.
MARIOS SAVVIDES: But the general face structure, that is pretty much well-maintained. All these features are there from this.
MILES O'BRIEN: What about 12 pixels between the eyes, the kind of image you would typically get from a CCTV camera?
MARIOS SAVVIDES: So let's see what happens when we run our algorithm. OK. Wow.
MILES O'BRIEN: Not bad.
MARIOS SAVVIDES: Not bad. So we went from this image to this image.
MILES O'BRIEN: Yes.
MARIOS SAVVIDES: And if we compare it to the actual high resolution, again, the eyebrow structure is there, the nose, the eyes, the mouth. We have lost the high-frequency texture because there is no information. But the structure of the face is there.
MILES O'BRIEN: How low can it go? What about six pixels from eye to eye?
MARIOS SAVVIDES: So from this to get this, that's amazing, and that's how computers can be better than human minds, because I could not extrapolate this information from this.
MILES O'BRIEN: Boy, I should have shaved, huh?
Anyway, you're really getting every whisker.
The secret is machine learning and pattern recognition. Instead of trying to tell the computer how to turn a few pixels into a recognizable face, Savvides simply shows it thousands and eventually millions of examples of the same faces in very low and very high resolution. Over time, the computer identifies patterns and makes connections that humans cannot. It learns.
MARIOS SAVVIDES: That's where computers can be smart, and that's where it can do things that our human brain cannot do, because it can learn the relationship of what a low-resolution, degraded-face image is, and what the high res. When you give it enough data to learn that relationship, then it can actually build an algorithm to extract the high-res face that you can feed into such systems to try to give you a match.
Well, if you use the periocular, we're not looking at the mouth.
MILES O'BRIEN: So, no expression?
MARIOS SAVVIDES: It doesn't affect it all.
MILES O'BRIEN: Expression can be a problem for you, right?
This is not all they are working on here. They are looking for ways to identify faces simply by focusing on the eyes or even just the eyebrows. They are predicting age better than a carnival barker. They are working on putting facial recognition software into smartphones, and they are developing sophisticated infrared cameras that scan irises.
COMPUTER VOICE: Identified. Welcome, Miles.
MARIOS SAVVIDES: When law enforcement has nothing to work on, have no leads, any lead is something. So there's a chance it may be wrong. It was wrong, OK, but it was a lead. But if it was right, if it was able to get you an image in that top 50, then that's huge. And this is the thing. How can you give the investigators something to work on?
MILES O'BRIEN: Facial recognition may not measure up to the movies just yet, but it is coming soon to a police department near you.
-t