1404: Alex Davies | The Grizzly Truth About How Safety Gets Tested

1404: Alex Davies | The Grizzly Truth About How Safety Gets Tested

This transcript is yet untouched by human hands. Please proceed with caution as we sort through what the robots have given us. We appreciate your patience!

Jordan Harbinger: Welcome to the show. I'm Jordan Harbinger. On The Jordan Harbinger Show, we decode the stories, secrets, and skills of the world's most fascinating people.

Our mission is to help you become a better informed, more critical thinker through long-form conversations with a variety of amazing folks, from spies to CEOs, athletes, authors, thinkers, performers, even the occasional drug trafficker, investigative journalist, or legendary Hollywood actor. If you're new to the show or you want to tell your friends about the show, I suggest our episode starter packs. These are collections of some of our favorite episodes on topics like persuasion and negotiation, psychology and geopolitics, disinformation, China, North Korea, crime and cults, and more.

That'll help new listeners get a taste of everything we do here on the show. Just visit jordanharbinger.com/start, or search for us in your Spotify app to get started. Today on the show, for you to trust an airplane, a child-resistant bottle, or a supposedly bear-resistant cooler, somebody has to try very hard to break it.

Sometimes that somebody is a grizzly bear named Kobuk the Destroyer. Today journalist Alex Davies takes us inside the strange world of product testing, why passing a safety test doesn't always mean surviving the real world, how blaming user error can hide a design failure, and what it takes to build things around actual human behavior, including ours, unfortunately, we're a difficult species to warranty. Here we go with Alex Davies.

Before we get into airplanes and AI and nuclear reactors and crash test dummies, tell me about Kobuk the Destroyer, because this is essentially an orphaned grizzly bear, and he gets a job in product testing, which, the surprising part is bears have jobs, sometimes.

Alex Davies: Kobuk was a grizzly cub who was born in the Alaskan interior, and his mom was a classic problem bear, which is the problematic, so to speak, term for bears who are really comfortable around humans and have a drive for human food, and bear-human encounters tend not to go super well. People can easily get hurt, and so bears who are comfortable around humans are often relocated or euthanized.

What happened to Kobuk's mom was she led her cubs onto an Alaskan rancher's farm and started killing his free-range chickens for food, and the rancher killed Kobuk's mom and one of her cubs, but Kobuk and Akina, his sister, were the two cubs who survived, and they were relocated to a bear sanctuary in Montana, just outside of Yellowstone National Park, and this bear sanctuary is basically a home for bears who have gotten too comfortable around humans and thus aren't super safe in the wild, but as problem bears, they also have jobs.

Jordan Harbinger: They've got to earn their keep, I guess, in the refuge.

Alex Davies: They have to earn their keep.

They have to sing for their supper. And the best way to stop creating problem bears is to keep bears out of human food in the first place, so that's why when you go to a national park, they have those garbage cans that you have to slide your fingers in to pop them open, and if you go backpacking, you usually have a bear canister that's impossible for me to open, at least.

Jordan Harbinger: Were you ever a Boy Scout?

Did you ever do that?

Alex Davies: No, but I've been a backpacker.

Jordan Harbinger: So I remember back in the, I guess it was 90s, a bear bag was, you take a bag, you put your food in it, and then you sling a rope around a tree branch, and the bear supposedly can't get it that way, but I don't know.

I've never had to use that, and it just seems too simple that if that actually worked, you wouldn't have bear-proof food boxes. You would just sling a bag over a tree branch. So maybe that doesn't work?

I don't know.

Alex Davies: I don't know if it works that well. I tend to do both.

I take my bear canister, it's screwed on really tight, you have to hit these multiple tabs at the same time to open it, and then I put it in a bag, then I sling it up in a tree.

Jordan Harbinger: I see. So they weren't totally off.

So bears, I guess, not having opposable thumbs or whatever they're called, supposedly they can't open these things.

Alex Davies: Yeah, and you have to hit it at exactly the right spot, so you have to be able to read the markings.

Jordan Harbinger: I'd starve to death, but yeah, that's why I don't backpack.

Alex Davies: So the bear canister I use is certified as bear-resistant. And the way you get that certification is if you're a manufacturer and you've got a new garbage can, a new cooler, a new bear canister design, and you say, this is going to keep bears out, the way you prove it is you bring it to this bear sanctuary where Kobuk lived, and you fill it with honey and fish and treats and all the kinds of things bears like, and you give it to one of the bears. And if one of these bears can't open it within the space of an hour, and that metric is called bear contact time, if he spends one hour of bear contact time trying to crack open that thing and can't, it is officially certified as bear-resistant.

Jordan Harbinger: An hour's a long time, because if you think about safe cracking, which I don't expect you to know much about, I also don't, but I think there's different levels. There's like one, two, three, whatever. The top home safe you'd get is, I don't know, it's like, let's say it's two or three.

That's 15 minutes, right? With a safe cracker. So if a bear gets an hour, that's a, yes, it's a bear, not a human, but that's a decent amount of time for something to safe crack, essentially a food box.

Alex Davies: And it's time for the bear to try a bunch of different things. And the reason we do testing with actual bears is that the mechanical tests that ran before this program was established about 20 years ago, dropped a metal weight on it and used machines to mimic the physical force of a bear. But what they don't mimic, what a spiked block can't mimic is the intelligence of a bear.

That a bear is going to try different things. They're going to pump up and down on it. They're going to claw at it.

They're going to try different angles because they're really motivated to get in it. The reason Kobuk got the nickname, the destroyer, is because he was the best bear at cracking things open. And I talked to the human who actually ran this program and he said, Kobuk could get into just about anything.

And so manufacturers would bring these new designs, these new garbage cans, and he'd figure out like, I can't fit my whole paw into this slot, but if I twist myself around like 180 degrees, I can just get one of my longer claws in there and pop it open. And so manufacturers would get so frustrated because they thought they had something bear resistant, but it wasn't Kobuk resistant.

Jordan Harbinger: So he's a destroyer of engineers' egos, basically.

Alex Davies: Exactly.

Jordan Harbinger: Because he's not destroying the container. He's just opening it like I would.

It's just taking longer and it's a little more clumsy. But yeah, he's destroying the design.

The guy's over there patting himself on the back because the weight that dropped on it couldn't open it or whatever.

And then Kobuk is like, well, I'm going to go ahead and make you redesign this entire thing. Because I guess once they open your prototype, you're back to the drawing board, essentially, right? Like you've got to add some element in there that they can't do at the same time.

Like maybe he could do the flip turn and open it, but now he's got to push a tab down. But you've got to redesign and re-manufacture or whatever the whole prototype so that he can't. This is pretty interesting that he figured this out.

It seems like hubris to go, oh, we'll just have machines mimic the force because bears are so dumb, all they are is strong. I don't know much about bears, but they're not just dumb meat machines. No, they're smart.

Obviously, they're smart enough to figure this out. So it's just human arrogance that says, oh, we just need a machine that's strong enough to smash this thing like a bear would. They're not going to figure out how to use the latch.

And Kobuk just proved everyone wrong.

Alex Davies: Exactly. And that's what makes it good testing is that instead of just the machine that says, well, a bear will bite it, and this is about how strong a grizzly bear bite is, they say we need a real bear to test this because that's what the product is going to face in the wild.

Jordan Harbinger: So there's a problem though, right? Because Kobuk is one bear. A few bears passing something or failing something test-wise are opening or not open.

That's not nature, right? So there's this Yellow-Yellow. He's an even smarter black bear.

And this guy, he was a genuine safecracker bear from the sound of it.

Alex Davies: Yellow-Yellow was a female actually.

Jordan Harbinger: Oh, okay.

Alex Davies: She was really smart. She was a wild bear and they called her Yellow-Yellow because at some point she had been tagged by someone who was just tracking, by some agency that was tracking bears in the wild. And she did manage to crack open bear canisters that had been certified by bears like Kobuk.

So there are limitations to this test program, which, as you said, Kobuk is one bear in one hour. And so this product managed, and maybe Kobuk or Akina or the other bears in this program, maybe they're smarter than bears in the wild, or maybe they're dumber than bears in the wild, or maybe that day in that hour, they didn't have the right idea that a bear in the wild would have. And this is a central tension in all product testing is that you want to create a test that's as similar as possible to the real world, but you also need a test that's workable, that's practical, that's not super expensive, that's repeatable.

So you actually get reliable results and you can't just find a bear in the wild and throw them a thing and see if they'll open it. It's completely impractical, right? How are you going to go back to a manufacturer and say, well, we couldn't really film it, we gave it to him, he ran off with it.

A bear in a sanctuary is a controlled environment. But what you're sacrificing is some of that variability among bears.

Jordan Harbinger: Yeah.

I assume bears learn in similar ways as other mammals. So if they see 100 containers, they have 100 different ideas on what to try. Whereas a bear in the wild possibly has three different ideas or no idea, depending on how many human food containers they've come across in the past.

Alex Davies: And that's kind of what we're banking on. But a bear like Yellow-Yellow might have just one better idea and figure out how to crack it open. Or maybe she got lucky and just kind of hit the thing or it wasn't sealed properly or something like that.

Jordan Harbinger: So the reason we're doing the show is because product testing, product testers are kind of the invisible people who make the modern world work, or at least work in ways that don't kill us all immediately, because we lionize the inventor, right? We know Jobs. We know Edison.

We know the Wright brothers. We don't really know anybody whose job is to say, oh, this is a cool invention. All right, let's see if this kills you when you use it or injures people in large numbers.

And that's what product testers are. I mean, maybe they just make sure the thing works, but the products we actually care about, like airplane jet engines, those are the ones that I want to make sure those people really know what they're doing. How did you get interested in this?

Because it's kind of, first of all, like, wow, this exists. And then you became obsessed with it.

Alex Davies: So I was a reporter for Wired magazine for a while, and I covered the transportation field.

And so I'd cover lots of new cars and bikes and all sorts of different products that got people from one place to another. And I realized kind of gradually that a good way to get an engineer to tell you more about the product than what they have written down on their spec sheet or what their PR person has told them to highlight is to ask them, well, how did you test it? Or what version of this didn't work?

And I found that talking about testing and design is a really good way to get into the why of the thing, why the final product looks like it does. And I really enjoyed writing these individual testing stories. But as I started to think more about testing as a field, none of those stories actually gave me the room to talk about those tensions, like I was talking about with Kobuk, of similarity versus workability or who test engineers are on a larger scale.

And so I went slightly crazy and decided to write a whole book about it.

Jordan Harbinger: I'm fascinated with these subjects that people don't know exist and then turn out to be really interesting. Stuff like shadow fleet oil tankers, and it's like, there's a whole rabbit hole to go down with this kind of thing.

And airplane testing or just product testing in general, you can kind of reduce it to five steps. I'd love to hear about these, because underneath all the frozen chicken cannons and the crash tests and the burning of safes and angry, hungry bears, most testing just follows the same five step process. Can you walk us through those in plain English, how it works, what people are looking for?

Alex Davies: The five steps to running a good product test actually look a lot like the scientific method, if you remember that from high school. The first step is to what I call observe and imagine what might happen to a thing. So you've got a bear canister and you've got to think, well, a bear is going to get this.

A bear is going to try to claw it. A bear is going to try to bite it. A bear is going to pump up and down on it.

And your task is really to think of everything that might happen to it. Step two is set your standard. So how bear resistant do you want this thing to be?

And on a similar note, how human openable do you want this thing to be?

Jordan Harbinger: You don't want to get too ridiculous, right? Like it needs to be able to withstand a family of bears all pulling with maximum strength in opposite directions.

And it's like, okay, that's probably never going to happen. And now it weighs 180 pounds, so you can't use it.

Alex Davies: Right.

And it's no longer usable for backpacking. Whatever tree you sling it up in is going to come down. Another good example of that is child resistant medicine bottles, which are tested by actual kids.

You get a bunch of kids together in a panel and you give them five or 10 minutes to try and open it. And there are sugar pills inside or candy, something that motivates them to open it. And the test, the official regulations say that 85% of kids have to fail to get into it.

And you think 15% of kids able to open a medicine bottle, that's a lot of kids. That's a lot of kids eating grandma's Percocet. Yeah, that's not good.

But they have to strike the balance because grandma needs her Percocet and grandma has arthritis and needs to be able to open it. So a lot of standard setting is finding that balance between what's the right level of strength or what's the right level of difficulty or endurance. So step three then is create a test that recreates those conditions that you talked about in step one.

Get a grizzly bear. Give them the bear canister. Grab a five year old.

Give them a pill bottle. Step four is that idea of workability. How are you going to make sure this test is repeatable?

Because if I'm testing different versions of a medicine bottle or of a bear canister, I want to make sure that those tests produce results that are actually comparable, that aren't random so I can fairly compare them and understand them. And so that's why with both the bears and the kids who are actually doing very similar work, they're filmed carefully, there are strict rules, there are strict metrics. For Kobuk, the metric was bear contact time.

It was one hour of bear contact time and that hour counts or every minute Kobuk is biting or mauling or stepping on a thing, that counts as bear contact time. But if he's just sitting there licking the outside, that doesn't count toward bear contact time.

Jordan Harbinger: I was going to ask what if he's like, all right, I'm thinking or I'm taking a break or I look, there's something else that got my attention.

Is the clock running? Because that would just mess these tests right up. Nope, the clock stops.

He has to spend an actual hour trying to get into it.

Trying to get my kid to focus for an hour and he's seven years, that's very, I mean, I don't even know if it's possible actually right now to get him to do that. For me, that would be a challenge.

So bears are very determined and very focused, I guess. That's a lot of motivation to sit there for an hour doing something, especially when you're out in nature and maybe there's other sources of food. I mean, that's pretty impressive somehow.

Alex Davies: Yeah, and the bears are really motivated to do and that's why problem bears, these human familiar bears are actually good bears to use for these kinds of tests because they know what human food tastes like and they really want it.

Jordan Harbinger: Yeah. Calorie dense, honey fish.

I mean, the honey fish mixture sounds absolutely revolting and right now having this cold, it kind of makes me just want to immediately vomit. But you know what? Bears are, I don't know.

They just need those calories, I suppose.

Alex Davies: So often what's in there, I think it's like granola bars and freeze dried beef stroganoff. Actually that sounds not so bad.

Jordan Harbinger: Is this a universal engineering methodology or is this a framework that you kind of came up with for wildly different kinds of testing because it makes the book a little bit more legible?

Alex Davies: It's a framework I came up with, but just by observing lots of tests and I said, okay, well you can see these. So it's not necessarily that all these things are done in this specific order.

We didn't even get to the final step, which is step five, which is simply run the test, judge the results.

Jordan Harbinger: Let's talk about the Boeing 777. This is 1995.

I mean, now we know there's more, the Max or whatever.

Alex Davies: Yeah, the 737 Max.

Jordan Harbinger: Yeah, we'll probably get into that in a bit here, but there's a brand new 777 inside a hangar.

Hundreds of people are watching. Again, January 1995. Cables attached to the wings and for some reason somebody has a yellow rubber ducky on the wingtip.

So what's going on with this plane and what's the ducky about here?

Alex Davies: So this is the wing bending test and this is something that Boeing does to all new plane designs and what it does is you put the plane in a hangar and you attach all of these cables to the wings and those cables winch upward. So the wings that are normally like this go up, up, up, up, up, and you see how far they can go.

And there are, if we go back to that five steps of testing, the first thing those engineers did, or engineers decades ago did because they're following set certification rules, is what is a plane going to face in the sky? What's the worst storm going to do? Because you know you need plane wings to flex a little bit.

If they were completely stiff, they'd be much more liable to break off and you want them to be pretty flexible to handle pretty rough storms. And they said, okay, this is the number of degrees we think in the most severe storm this plane will ever face. Let's say it's a 50 degree bend, which is pretty extreme.

It's probably not that. And then they said, we want to go further than that. There's this idea called the factor of safety, which is where you take what a thing is going to handle and you multiply it.

So a factor of safety of two is if you think a 150 pound person is going to sit in this chair and that's the limit, you're going to make it strong enough. A factor of safety of two means you could actually support a 300 pound person sitting in this chair. So for aviation, because weight is a concern, the factor of safety for everything is 1.5. So they say the plane wings, 100% is the plane wings bending as far as you think they'll ever have to in flight. And then they have to go 50% further than that. But the 777 was a funny plane for Boeing in that it was the first plane that was designed in a computer completely. And so they said, we have all of these computer tests, but we, we really want to see how far it'll go.

So that day in January, 1995, they were going to bend the plane wings until they shattered.

Jordan Harbinger: Just no matter what, like even if it's a ridiculous 80 degree, whatever angle, they're just going to do it until the thing explodes.

Alex Davies: They were just going to go until it shatters.

And one of the Boeing engineers put a rubber ducky on one of the plane wings just for fun.

Jordan Harbinger: Oh, okay. I thought there was some clever function.

It was now we're just going to film this. So I thought I'd do that.

Alex Davies: Yeah, exactly.

So it was part of a documentary about the making of the planes. I said, yeah, this is just a bit of color. And so the plane wings get to a hundred percent of what you think they're going to handle.

They get to 150% and the people in the crowd kind of applaud because they know it's going to get there. And then it keeps going up and up and up. And then ultimately it's like at 154% that they shatter into a million pieces, fly all over this hangar, including the rubber ducky goes flying, but actually the rubber ducky lands upright on the floor.

Jordan Harbinger: Years of engineering school and your final boss is a bear with the munchies. Here's a word from our sponsors. We'll be right back.

This episode is sponsored in part by AT&T. If you've done my free Six-Minute Networking course, you know I'm big on keeping in touch. It doesn't have to be a whole production.

Send somebody a quick text, share a photo that reminds you of them. That's part of digging the well before you're thirsty, building relationships through little everyday conversations so you're not only reaching out when you need a favor. And AT&T has a deal to keep those connections going right now at AT&T, you can get the new iPhone 18 Pro for $0 with eligible trade in.

It has the ultimate pro camera system and a big leap in battery life so you can turn this deal into a long lasting connection. Share a photo, check in on an old friend, follow up on something they told you last time you talked. You don't need a big reason to reach out.

Just remembering what's going on in someone's life is a pretty good one. Learn how you can trade in your phone for the new iPhone 18 Pro at AT&T. Connecting changes everything.

Required trade in of $230 or more on an eligible plan. Terms and restrictions apply. Subject to change, visit an AT&T store for details.

This episode is also sponsored in part by BetterHelp. If you've thought about trying therapy with our long term partner BetterHelp, insurance options may now be available nationwide. With an average copay for eligible members of just $23 per session.

World Mental Health Day is October 10th, but apparently nobody told your brain. Stress does not consult your calendar, burnout never sends a meeting invite, and that problem at work can suddenly convene a full board meeting in your head at about 11:47 PM. You also don't need a dramatic turning point to talk with a therapist. Therapy can be a place to examine the smaller things occupying a suspicious amount of mental real estate.

A recurring relationship pattern, a decision you keep circling, or that persistent feeling that something is just off. If you've been waiting to start therapy, BetterHelp works with more than 125 health plan networks, and eligible members have an average copay of just $23 per session with insurance. Don't have insurance or live outside the U.S.? You can still get 10% off your first month. Visit BetterHelp.com/Jordan to explore your payment options. That's BetterHelp.com/Jordan. When you use a code, I don't get a cut of the sale, but companies do see you listen to and you use the ads, and they keep shoveling money in our direction, so if you decide to sign up or buy something, please use our code.

It's usually Jordan, but not always. They're always on the deals page at JordanHarbinger.com/deals. It's a double win.

You get a great deal and you help keep the show going strong. Thank you for your support. Now, back to Alex Davies.

Seemingly, this is an idea that you have to do. I don't care how good the computer simulation is. This thing better be tested in reality before I get on there.

So was the idea to just see if the digital predictions matched reality at all, like, is that why they bothered to do the computer predictions in the first place? Or did the computer predictions get you 80% of the way there and then reality dictates whether you can use the design?

Alex Davies: I think the computer predictions say this design is definitely going to work, but you want to be sure, especially if this is the first time you're doing it in the computer.

Yeah, the computer's kids are not on this flight. Right. You want to be sure, and also there's a little bit of theater in it, and there's a little bit of theater in a lot of testing.

That's why you see car crashes in commercials, because there's a little bit of theater. So look at what this thing can handle.

Jordan Harbinger: Right.

Look at the slow-mo of the crash dummy with the little yellow-black circle on the side of his face, smashing into the airbag, smashing into the brick wall. You remember those seatbelt commercials as a kid where the crash test dummies were talking and they're like, I don't need to wear my seatbelt, I'll be fine. Those were so creative, right?

Yeah, those PSAs. I think the slogan was don't be a dummy, wear your safety belt, because you actually had to convince people to do that in the 80s and the 90s. Oh gosh.

We have come a long way, haven't we? I remember my friends being like, you don't need this, this is ridiculous. And people's parents were like, ah, we don't bother with that in my house.

It's like bike helmets. Did you ride with a bike helmet when you were a kid?

Alex Davies: Yes.

Jordan Harbinger: Oh my god, we didn't. In Michigan, if you had a helmet, everyone made fun of you, which is ridiculous. I would never let my kids ride without a helmet.

I don't even ride without a helmet now. I don't do that just because I grew up that way. It was just, oh no, we were total idiots back then.

Alex Davies: Yeah, I ride without a helmet sometimes and I think, this is really stupid.

Jordan Harbinger: So dumb. It's like riding in a car with no seatbelt.

Hey, it costs you virtually nothing to do this. You already have it. It's in your garage.

You moved it when you got your bike out and you just didn't put it on because you don't think it looks cool. I mean, it's really... Anyway.

So the factor of safety thing, the factor of safety of two or three, so if you build a bridge and it's got a factor of safety of three, which means, I don't know, three times as many cars can fit on it or something like that, is that just a judgment call? Or are there regulations that say, this thing needs a factor of safety of three? Because it's not like they're coming down on stone tablets.

Somebody's figuring out what you need.

Alex Davies: It's both. So a lot of these things are written into certifications, like a crane has a factor of safety of five, I think, and a crane in a steel mill that's holding molten metal has a factor of safety of like eight or nine.

Jordan Harbinger: Yeah, good. That seems like a good idea.

Alex Davies: Elevator hoist cables are like 11 or 12.

Jordan Harbinger: That's good to know. You know when you fill the elevator and it goes, beep, and then the door's closed and you're like, oh great, we're all going to die now because this guy got on. And it's like, nah, you're good.

You could probably fill the thing with metal boxes.

Alex Davies: Exactly. And so they say the weight limit, and the weight limit is not really the weight limit most of the time.

It's because of that factor of safety idea. It's probably like eight times that.

Jordan Harbinger: That makes me feel a little better.

Yeah, okay. So who decides how much margin is enough? There's got to be some equation that's like, okay, a crane that's holding a car at a junkyard, three is good.

A crane that's holding, like you said, a giant, I don't even know what you'd call it, like a giant cauldron of molten metal with a bunch of humans somewhat nearby, where if this thing falls, the entire place is destroyed and a bunch of people die. Let's add a little buffer to this. Somebody's making these judgment calls.

Alex Davies: They're judgment calls, and usually they're compromises. They're compromises often between government regulators, consumer advocates, the actual industry who's making this stuff. And so someone wrote a whole history of how aviation ended up settling on that 1.5 figure. Because at the beginning, one of the Wright brothers said, I think the plane should be like five times stronger than it needs to be. That's a factor of safety of five. And then they just kind of like waffled around.

It was two for a while, it was one for a while. And ultimately this history was like, people just settled on it over time.

Jordan Harbinger: That's not super comforting though, Alex.

Like, hey, oh, you know what? They just kept loosening this thing up over time. And then they decided, you know, it's fine.

It's like, no, no, no, no, this isn't an airplane. I don't like this thought process, man, at all.

Alex Davies: Yeah, that 1.5 figure I found scares people. Yeah. I'm like, it's one and a half times stronger than it could ever need to be. But still, you want it to be like two or three.

Jordan Harbinger: Great, but like a crane is eight times stronger, an elevator is 11 times stronger. I fly in airplanes less than I take elevators fine, but 480 people die if this thing breaks and there's only like a 50% margin. I don't love that.

I don't at all.

Alex Davies: Yeah. But you know, planes are really safe.

And the thing is, it's a compromise because that plane needs to get up off the ground. It has to be light enough, right? You could make it a factor of safety of 10, but it would be lead.

Jordan Harbinger: It's a bus.

Alex Davies: Yeah, it's a bus. It's never going to get into the air.

Or it's going to use so much jet fuel that, you know, then like you're dealing with other, it's going to be. That's true. Hugely inefficient, hugely expensive.

Jordan Harbinger: You have a wonderful, wonderfully, I should say, mundane way of explaining the structural engineering thing. You got the grocery store question, single bag or double bag? Deciding whether to double bag your milk is basically amateur structural engineering.

Alex Davies: Yeah, a double bag is a factor of safety of two. And so what I talk about in the book is these things are negotiations and that's a negotiation just with yourself. Although sometimes the clerk, grocery clerk is like, you should double bag this.

But you're thinking about all of these different factors, right? Did you drive to the store and you've just got to walk 50 feet to your car or did you walk three blocks? So you want it to be stronger.

Are you buying grape juice or are you buying a fine wine? Did you want single bag and the milk fell through and you made a huge mess? And it's experience, it's judgment.

It's your own innate sense of risk. But when you talk about an actual serious factor of safety, it's a bunch of people doing that together and figuring out, kind of triangulating to one number that works.

Jordan Harbinger: I guess it's a relief knowing that back to the airplane thing, even that factor of 1.5, we have very few aviation disasters, even at one point. So if something is 2, 3, 5, 6, 7, 8, that really is just a lot safer, right? It's hard for my puny human brain to sort of wrap around being twice as safe. But actually, that's a huge, huge margin, right?

If you think about it.

Alex Davies: It's a huge margin. And the planes that crash don't crash because they exceeded that margin.

They crash for other reasons.

Jordan Harbinger: Yeah. Other reasons.

In fact, speaking of other reasons, tell me about the de Havilland Comet, the world's first commercial jetliner. This thing just started exploding in midair, which is terrifying.

Alex Davies: Yeah.

So it didn't so much crash as it exploded.

Jordan Harbinger: It exploded. Totally different.

Don't worry, everybody. It didn't crash. It blew up.

Everyone died in the air instead of on the ground.

Alex Davies: So the de Havilland Comet was one of the, I think of as like the British Empire's last big swings.

Jordan Harbinger: Oh, man.

Alex Davies: They're like World War II kind of eroded everything. Like, no, we're still great engineers.

Jordan Harbinger: We're going to come back with this amazing invention.

Oops. Yeah.

Alex Davies: Yeah.

And it was the world's first real passenger jet, a jet airplane, as opposed to a propeller airplane. It would fly much higher than planes before it. And ultimately that was a role that ended up being taken over by the Boeing 707, which is what we remember.

But the Comet was first and it was this beautiful plane. And the thing they had to figure out was because it would be flying higher than previous planes, it was going to fly up above the weather where jet engines were much more efficient. They needed to figure out how much pressurization this plane could take.

Because if you're flying all of a sudden at 30,000 feet or around there, you need to pressurize the inside of the plane. So it's not like you're breathing air at Mount Everest.

Jordan Harbinger: Right.

Alex Davies: Yeah. That would be an uncomfortable flight. You'd probably end up unconscious.

Jordan Harbinger: Yeah. Or dead, probably. Because don't they need oxygen on Everest?

I don't know. I'm not a mountain climber, but I've seen the tanks.

Alex Davies: I think the people with oxygen do better on it than the people without it.

Jordan Harbinger: The people who can breathe somehow have an advantage.

Alex Davies: Yes. A fun point.

Even in planes today, they don't pressurize it to create the effect of sea level. They pressurize it to about 5,000 feet, which is about the elevation of Aspen. So if you're really tired after flights, that's probably one reason you're not getting your normal level of oxygen.

Jordan Harbinger: I don't know about you. As soon as I get up in a flight, as soon as they're like, we're at cruising altitude, you can get out your computer. And I'm like, all those big plans of getting out my computer and doing email.

I just need a quick nap. And then I have however many hours of fake, not actually restful, air quotes, sleep on the airplane. Maybe I can watch part of a movie.

Even then, I fall asleep. I mean, I would kill to be that sleepy at any time during my normal life during bedtime. But alas, it only happens at 9 a.m. on a Tuesday when I'm in the air at 5,000 feet.

Alex Davies: Move to Aspen, I guess.

Jordan Harbinger: But then you'd acclimate. Yeah, I guess.

Yeah.

Alex Davies: First week, sleep like a baby. The de Havilland engineers need to figure out, they know they need to pressurize their plane and they're going to need to pressurize it to a level that no plane before it had dealt with.

And they need to do this on the ground. And so what they do is they take a section of fuselage and they put it in this huge water tank because they could pressurize it with air. But once the plane gives and explodes, it's going to be really impractical to figure out where it gave.

And so by putting it in a tank of water, so you're surrounding it with water and then inside you're pumping in more and more water to recreate that pressurization effect. And so when it does buckle, the water around it holds it kind of intact so you could see where the give is. So what de Havilland did was they said, OK, this is the plane.

Let's figure out how many cycles of pressurization it can take. Pressure, unpressure. Because they know metal fatigue is a thing where metal gives out over time.

But in the 50s, that's pretty much the extent of the knowledge about metal fatigue.

Jordan Harbinger: This is why when you bend a wire and then you bend it again, you bend it again, you bend it again, and eventually it just snaps at the place where you're bending it, right?

Alex Davies: Exactly.

Jordan Harbinger: Yeah. Like a paperclip bending.

Alex Davies: At the time, all they knew about metal fatigue was that it existed and they couldn't really predict it.

They didn't have the science. And so what they did was they put this plane in the tank and they just pressurized it and depressurized it and pressurized it and depressurized it and counted the number of cycles that could do that. And when it ultimately gave after something like 13,000 cycles, they said, OK, let's make sure to retire this plane at like a thousand flights.

So this thing is super safe. Right. Right.

We know it can do 1500 pressurization cycles. So after a thousand, we're going to stop flying it just to be safe. There's this really tricky thing, though, that happened, which was the other thing they did before the repeated cycles.

They did the max pressurization test, where it was one test to figure out just how much it could take. Could it take 2x the amount of pressurization it takes? Or could it take 2x or something like that, the amount of pressurization they expect it to need to do?

So again, that's that factor of safety idea. And they did that, and then they used the same section of fuselage for the repeated tests. What they didn't know was that first test actually slightly changed the structure of the metal.

Jordan Harbinger: Oh, how?

Alex Davies: I think it closed some of the cracks. And ironically, it made it a little bit safer.

Jordan Harbinger: Huh. So it just kind of smashed everything together and made it stronger?

Alex Davies: It's something like that.

I don't get too far into the metallurgy science.

Jordan Harbinger: Yeah, I was just curious because it seems like—

Alex Davies: To keep it high level.

Jordan Harbinger: It just seems like an obvious oversight, but that's because we know what metal fatigue is and because you've led in such a way that we know this eventually failed.

So it seems like they should have thought of that. But then again, this is, what do I know?

Alex Davies: Well, the thing is, they just didn't have the science.

And I talked to a metallurgist who actually studied this plane about it. Today, you'd look at that and go, yep, absolutely. I know that's going to happen.

But they had no idea.

Jordan Harbinger: Yeah, oof.

Alex Davies: They were trying really hard to be really safe with this plane because some of the first passengers were like Prince Philip.

Jordan Harbinger: Right.

Alex Davies: And Princess Margaret. They did not want it to explode.

Jordan Harbinger: Did the first flight explode or just after a while it was— Oh, it just had less tolerance. So instead of exploding at 13,000 cycles, it was like 500. Yeah.

Oh no, that's terrible. That's really bad. Okay.

Okay.

Alex Davies: You don't know the exact— The explosion point was much lower than they thought. Much lower than the tests.

They did because as best as they tried to make those tests really similar to the real world, they were missing that one key point of information. And so the Comet flew. It was really successful for a year or two.

Fancy royalty flew on it. And then it started exploding. There was one flight.

Jordan Harbinger: But this is so disturbing, right? Because it seems like, and this is a little hinted at in the book, so the earlier maximum pressure test kind of blunted microscopic cracks around rivets and other connectors and stuff like that. So it unintentionally delayed the fatigue in the test fuselage of the plane and produced false confidence.

They gave it a pressure treatment that they didn't consider a pressure treatment and considered a test. And that actually made it stronger. So in theory, you kind of had to do that to every manufactured fuselage and that it would have been a better product.

But the engineers had missed that. But this is one of the most disturbing ideas in the whole book, in my opinion, because that means that testing itself can alter the thing that you're testing. So how do you ever know that your test setup hasn't sort of quietly manufactured the answer that you wanted?

Alex Davies: You don't really know. Yeah, that's the scary part. There's this fundamental idea, right, that you think like, OK, this test showed that the 777 wings can bend to 154% of what you think they're going to need to do.

But the plane they tested on is not the 777 you're flying on.

Jordan Harbinger: Right.

Alex Davies: Yeah.

It's basically the same, but it's not the same. And the bear who opened or couldn't open the bear canister, Kobuk is not the same bear as Yellow-Yellow. And the five-year-old who couldn't open the medicine bottle is maybe dumber than your five-year-old.

Yeah.

Jordan Harbinger: So the problem is, in one case, a bear eats your lunch. And in another case, your grandson dies of a drug overdose in your kitchen or your whole family dies in an airplane disaster.

Like the consequences of some of these are just a little higher than I think most people are willing to tolerate. And I guess that's why they say aviation rules are written in blood. But is that an unavoidable consequence of innovation?

Or is that kind of a euphemism for like, oh, we didn't imagine hard enough before a bunch of people died?

Alex Davies: I think aviation is a little bit of a special case because plane crashes are so high profile. And the industry, unlike a lot of industries, is super invested in staying super safe.

Jordan Harbinger: Yeah.

Alex Davies: Well, I appreciate that. It was plane manufacturers who, early on, actually went to the government and were like, we need rules.

We need strict rules because we've got all these yahoos out here making planes that are blowing up. And it makes us look really bad. And no one's going to get in an airplane if they read about plane crashes.

Jordan Harbinger: Also, we know how to make it safer, we think, but it's going to be super expensive. And the competition is going to undercut us by being like, yeah, we just didn't test anything. And it looks like it's probably going to be fine.

I don't know. Here's your plane. It's way cheaper than the other guys.

It's like, no, no, no, no, no. We need to make this safe. And also, we need everyone to have to make it safer.

Take AI. AI is kind of trying to do that right now. But they're saying, oh, we need rules.

But then the criticism to that is, well, it's just the big guys saying we need rules because they don't want little startups coming and competing with them. So they want to set rules up so that the established players like Anthropic and OpenAI are the only ones who can afford to essentially be compliant with government regulations. So there's a little cynicism attached to it.

Alex Davies: And the rules make them look good, too, because they're like, look, we're cooperating with government and we do think there should be regulation and we can follow it. There's that idea of regulatory capture where the rules get so complicated that only a couple of companies have the might or the manpower or financial resources to follow.

Jordan Harbinger: Yeah.

Or they're writing them, right? Oh, we're going to do this thing that our company is uniquely suited to be able to do at really a cost-effective level. But good luck, guys.

Good luck out there.

Alex Davies: Yeah. Why do you think there are like three planemakers in the world?

Jordan Harbinger: That's a really good point. There's Boeing, Airbus, and who else?

Alex Davies: China has one, but I forget the name of it.

And there's Embraer, which is Brazilian, which makes little regional jets. If you're flying New York to Boston, you might be on one of those planes.

Jordan Harbinger: They don't make the top list, but somebody's making planes in Russia.

Because I flew on like an Ilyushin or something it's called. And I remember the plane was fine, obviously. But you know those little indicators that they cheaped out on something?

That's really scary when you're on an airplane, right? Because you're already flying to North Korea on a North Korean airline. So you're like, yeah, this is not the safest way to travel in the sky.

But the air conditioners were all blowing the condensation out, because it was so cold and it was humid. But it was dripping. So it was raining in the cabin.

And the flight attendants would come by with this little handkerchief type of thing, constantly wiping the condensation. But it was still raining on you in the seat. And I thought, this is supposed to fly internationally over large bodies of water.

And with the AC units, they were just like, eh, it's fine. And I was like, well, hopefully the people that made the outside are not the same people that decided what goes on the inside.

Alex Davies: My general rule is that if an airline is allowed to fly into or out of the United States or really into or out of Europe, it's good.

Jordan Harbinger: Yeah. Air Koryo, not allowed to fly pretty much anywhere. I think they fly to Malaysia and China.

And that's it. And around North Korea.

Alex Davies: Maybe there's a flight to Russia.

And that is not me saying they're not safe. But what I'm saying is like, yes, the US and Europe have the really good rules.

Jordan Harbinger: Right.

Alex Davies: Best rules, I think.

Jordan Harbinger: You can't fly this giant flying deathtrap over our cities is a good indicator that experts have looked at it and decided that there's a problem. So let's talk about foolproofing things, because it's actually maybe not the fool's fault is sort of your argument here.

You argue that a huge leap in safety came when engineers stopped asking, why are people so stupid? And started asking, why did we build something where a completely predictable human mistake becomes totally catastrophic and or lethal? So what is foolproofing in your definition?

Alex Davies: Foolproofing is making a product safe, even if the person using it doesn't use it the right way. And there's this concept called foreseeable misuse. It's a legal concept because American tort law is actually very consumer friendly.

And that's an evolution that happened over the 20th century. It got more and more consumer friendly and put more and more of the onus onto the manufacturers to make things safe for people.

Jordan Harbinger: It's called strict liability.

I remember learning about this in law school. Basically, if your tires explode, it doesn't even really matter. Oh, well, he was driving them a lot and he was driving them in hot weather and he drove them on a car that he modified.

It's like, OK, but they exploded on the highway. He was speeding a little. OK, well, you should know that people speed a little in cars that they modify in hot weather on a highway.

This should not explode and cause your car to roll over and kill you.

Alex Davies: So the classic example of this is you've got a paint can, right? You want to open it.

What do you use to open it?

Jordan Harbinger: Screwdriver or whatever, right? You pry it off.

Do they still do that? That's what it was when I was a kid. That's how I opened paint cans a couple months ago when we were repainting.

So that design is basically the same as it has been since the 80s, right? You get the top and you pry it off.

Alex Davies: Yeah.

And like they're theoretically might be like an official tool to open a paint can. But no, you're going to use a screwdriver, even though screwdriver is not made to do that. It's made to turn screws.

But screwdriver manufacturers know that people will use them to open paint cans. So if you use a screwdriver to do that and it shatters and throws shards in your eye, you've got a claim. That's foreseeable misuse.

Jordan Harbinger: Like, oh, you were supposed to turn screws with it.

Yeah. But you know, people pry things open with it. You should have designed it such that it can actually do that safely.

Alex Davies: Yeah. But if you've got some wax in your ear and you're thinking, oh, just throw a screwdriver in there and you puncture your eardrum. This is where foreseeable misuse gets a little squishy because you can imagine it.

It is theoretically foreseeable, but it's not really reasonable.

Jordan Harbinger: This is where the jury goes, nope, you're so dumb. We're going to let you just live with the consequences of this one.

This is extra dumb. And there's no pattern of it either.

Alex Davies: If a thousand people a year did this, you might have to design against it.

Although there are certain limits to a screwdriver that can't go in your ear. I don't know what use it would be.

Jordan Harbinger: That's a good point.

It would have to turn some really big screws. So is this why we have warnings on packages that are like, do not stick in holes on various parts of your body? And you're like, why?

It's a hairdryer. Yeah. Who did that?

Somebody tried it. And then they were like, yeah, hey, do not put inside the body. And you're thinking somebody put this inside their body.

I need to hear this story because this is a television remote for a Sony Bravia. Who put this in their butt?

Alex Davies: And there's this concept in safety engineering called the safety hierarchy of product design.

And at the base layer, the simplest, most ineffective thing you can do is put a warning on something. Don't put this in your ear. Don't toss it in water.

Don't touch the toaster when it's hot. Then the next level up is shielding the hazard. On a table saw, you're going to put a protector over the blade.

So it's that much harder to get your finger in there.

Jordan Harbinger: This is like hairdryers. You maybe don't use one.

I don't use one, but my wife uses one. And it's got a little label on it. And there's a photo.

So just so you know, a diagram. So you're sure. Don't use this while you're in the bathtub.

Because of course, you're drying your hair while you're in the bathtub. You just washed your hair. Why not?

Bad idea. If you drop it, you die. And then also on the plug itself, even though your house should have this, there's a fuse.

So that if it hits water, it instantly, in shorts, it's supposed to instantly turn that electricity off. Even though again, the outlet and your house should both have fuses. They just wanted to be extra sure.

So that's the shielding of the hazard there alongside the warning.

Alex Davies: And then the best level is the top one, which isn't always possible, which is removing the hazard altogether.

Jordan Harbinger: Right.

Alex Davies: And so I read this example from a safety engineer named Ralph Barnett, who used the Genesis story to do this, the story of Adam and Eve.

Jordan Harbinger: Okay.

Alex Davies: With the knowledge of good and evil in it.

And he said like, what God did was a warning. He said, don't eat this. That's all he did.

He stuck with the bottom.

Jordan Harbinger: But the apples should have been higher. And then it wouldn't have been an issue.

Alex Davies: Right. Well, that's shielding the hazard.

Jordan Harbinger: Right.

Okay.

Alex Davies: That's putting the apple higher in the tree or making the tree really thorny. Eliminating the hazard altogether is why is this stuff in a fruit to begin with?

Why don't you just keep it where it was?

Jordan Harbinger: That's a good point. You know what?

We're going to get some really thoughtful emails from religious people who are going to tell me something like, well, it was a test, dummy. That's the whole point. Just like having a hairdryer that kills you if you drop it in the water is a test of whether you know how electricity works.

Alex Davies: Right. So in this reading, God is a lazy engineer. And there's this term called warn and scorn.

Jordan Harbinger: Yeah.

Alex Davies: Where you warn someone not to do something, but you don't take any steps to stop them. And then when they do it, you scorn them or you commit them to a lifetime of suffering.

Jordan Harbinger: People are going to love your comment about God being lazy. He made the whole universe in seven days. All right, pal.

But I get it. The point of that was, of course, it's a test, I assume. I'm so curious what people, people are going to get a little offended by this.

Of course, we mean no offense. It's just a fun metaphor.

Alex Davies: No, it's a good example because everyone knows the story and you can see those three levels.

Jordan Harbinger: Child resistant packaging, an intelligence test that some toddlers pass and I routinely fail. Let's hear from our sponsors while I find a responsible adult to open my vitamins. We'll be right back.

This episode is sponsored in part by Cometeer. A while back, we invested in a fancy espresso machine and a grinder. Thinking we'd make incredible coffee at home.

We even bought the fanciest beans from our local roasters. Somehow it never really came out right. Now that the equipment is collecting dust, what we made just doesn't compare to Cometeer.

And Cometeer takes seconds. And here's why. Cometeer sources coffee from the best roasters in the world, like Onyx and Counterculture, brews it to perfection, like, you know, they actually know what they're doing, and then flash freezes it at negative 321 degrees Fahrenheit with liquid nitrogen to lock in peak flavor and freshness.

It comes on dry ice right to your door in recyclable aluminum capsules. Just melt it to make hot coffee, iced coffee, or espresso-style drinks like lattes in seconds. No machine, no plastic brew path required.

The former president of Nespresso invested after one sip. Having tasted it, I also get it. It's an entirely new way to make coffee and it's become our every morning routine.

Professionals handle the brewing. We get to enjoy the coffee. We even gifted it to friends, a busy college student who needs a study pick-me-up, and our cousin who just started a new job as a lawyer.

Try it yourself. Get $20 off at Cometeer.com/Jordan or use code Jordan. This episode is also sponsored by Paka. I just got back from a weekend in Vegas, as you can tell from my voice, and I have to tell you about the most essential thing that I packed, my Paka hoodie. It was so cold. I was really glad that I brought it. I wore it all weekend and felt cozy the whole time.

Jen was laughing at me because I kept talking about how much I love this hoodie, but seriously, so comfy, looks great, and you can really feel the quality. On the flight home, I pulled up the hood, settled in my seat, literally thought to myself, so glad I got my Paka. Paka makes outdoor and lifestyle apparel from alpaca fiber, one of the world's most sustainable natural fibers. Their best-selling hoodie is softer than cashmere, warmer than wool, and breathable.

It's thermoregulating, odor-resistant, durable, made to last. Basically everything you want in a hoodie. Each hoodie is made in Peru and hand-woven by artisans, honoring generations of knowledge and tradition and connecting you to where your clothing comes from.

Over 250,000 people have already picked up the Paka hoodie, and I get it. Mine has become an essential. I wear it, I love it, and you should try it too.

It would also make such a great gift. It's comfortable, high-quality, and something you'll actually use. If you've been thinking about leveling up your hoodie game, this is your sign.

To grab your Paka hoodie, go to www.pacaapparel.com. That's www.pacaapparel.com.

If you liked this episode of the show, I invite you to do what other smart and considerate listeners do. Take a moment, support our sponsors. They're the ones that make the show possible.

All of the deals, discount codes, and ways to support us are searchable and clickable on the website at jordanharbinger.com/deals. And hey, if you can't remember the name of a sponsor or find a code, email us. We're happy to surface codes for you.

It is that important that you support those who support the show. Now, back to Alex Davies. So warn and scorn is, so instead of actually preventing it, we warn people not to do it and then we make them feel stupid if they do, I don't know.

What's the point of that? It seems like you're still facing the same problem. I mean, if an airliner explodes and you go, we told you not to fly it that high and you still did it.

It's like, well, that's cold comfort to the 458 people who went down in that thing. And I'm sure Air China is really upset about it. But these people's families are way more upset, right?

The shareholders.

Alex Davies: Yeah. And I feel like you see pretty often after plane crashes, they're not necessarily the plane maker because I don't want to cast stones that aren't, I don't want to make accusations that aren't based.

But there's always a lot of talk about like, it looks like pilot error. It was human error. It's the pilots, they screwed up.

You know, they weren't paying attention. They didn't do this. They should have known that that warning light could break.

Jordan Harbinger: Yes. And we love to transfer responsibility to the victim and be like, hey, I told you. It's, I told you so, engineering.

Alex Davies: Yes, exactly.

Jordan Harbinger: And, but that's fine if the consequence is what your milk fell in your kitchen. You won't do that again.

It's not fine when the consequence is your entire family died in a plane crash, right? So let's talk about the B-17 landing gear because this was pilot error. But also, come on, guys.

Alex Davies: Yeah, this was happening during World War II where a shocking number of trained military pilots coming home from missions didn't deploy their landing gear.

Jordan Harbinger: Jeez.

Alex Davies: As they were coming down to fly and they crashed.

They would cry like they'd survive a whole mission being shot at bombing Germans. And they'd come home and they'd crash because they weren't putting down their landing gear. And it's because the buttons for the flaps and the landing gear were next to each other.

They were basically the same kind of button.

Jordan Harbinger: Oh, man.

Alex Davies: And it was just pilots making mistakes.

And I think these missions were obviously harrowing, long, dangerous. You come home. So it's not like you're totally fresh and going to hit the right button.

Yeah.

Jordan Harbinger: You're frazzled and you're glad to be alive. And you push the button, you lean back in your chair and hopefully relax, take a deep breath.

Meanwhile, you just moved your flaps instead of putting the wheels down and crashing to the ground.

Alex Davies: So this is where the military leans on the emerging science of human-centered design, which is basically a kind of foolproofing where you're thinking, I made this thing. How is someone going to actually use it?

And how can I make it harder to use the wrong way? It's a different kind of foolproofing because it's not just making it safer to use the wrong way. It's making it harder to make the mistake in the first place.

Jordan Harbinger: So it's square peg won't fit through the round hole.

Alex Davies: Exactly. And so what they did in the planes was they just changed the buttons.

So like one was a circle and one was a triangle. And they were more clearly defined and it effectively fixed the issue.

Jordan Harbinger: Yeah.

It's such a simple fix too. So it's a shame that people died having this problem. Morphine pumps had kind of another higher stakes or high stakes issue with those nurses accidentally killing patients with morphine.

Alex Davies: Yeah, because when you think about a morphine pump, the user is actually the nurse who's doing things like setting limits on what's actually getting pumped into you, how often you can hit the little button. And so actually, if you think about that little button that you hit, that's good foolproofing in a sense, right? The medical establishment doesn't want you to get addicted to morphine.

Yeah, I appreciate that. And so they put in this interlock that says you got it, you get a hit, but then you can't have another hit for an hour or two hours or whatever it is. But where they fell down, the makers of these pumps was that the actual interface to set all of these limits was very confusing.

And nurses who are typically overworked just would occasionally make mistakes.

Jordan Harbinger: Oh no.

Alex Davies: And people would overdose on morphine because they put in the wrong number or it wasn't clear what they were doing or you'd have to enter something three times.

It was just, they were terrible interfaces. And eventually someone experimented around with this and said, well, actually, I've created this new interface that's easier. It color codes things.

Things naturally read from left to right. You have to confirm a choice that has health consequences multiple times, but it's easy to go back if you made a mistake. And basically it was just designed with people in mind, how people actually use things.

Jordan Harbinger: So instead of user error, it's use error, right? Because user error is like, well, you screwed this up. That one's on you.

But use error is how do we make this impossible or way harder to do wrong?

Alex Davies: Exactly. And people who really work on this stuff much prefer the term use error.

It's a semantic thing, but it shifts the blame from the person to the product. Because here's the thing. You can't fix people.

Warnings don't work.

Jordan Harbinger: People make mistakes, but you can change a product. This is quite fascinating.

It seems like good engineering knows it will fail. You try to find the points that it will fail and you try and make those impossible for the user to do by accident, I suppose. Is that what they're aiming for?

Alex Davies: Basically. And that's where I like the story of SawStop, which is this guy who made a table saw that can't hurt you.

Jordan Harbinger: Oh, yeah.

Alex Davies: Because Americans lose two hands worth of fingers to table saws every day.

Jordan Harbinger: Oh, my God, that's really gross.

Alex Davies: They're very dangerous.

And because they're common and affordable and a lot of home woodworkers use them. And the injury rates are really high. And so, obviously, there are warnings.

Don't stick your finger in here.

Jordan Harbinger: Right.

Alex Davies: You know, hold it from a distance, wear safety goggles.

The warnings don't work that well. And there are shielding the hazard thing. There's often a plastic guard over the blade.

But a lot of woodworkers say, actually, this makes it harder to use. So they take it off. So there is this guy named Steve Gass, who is an amateur woodworker and professional patent attorney, which actually come together nicely in this story.

He was using his table saw and he thought, this thing is eventually going to hurt me. Could I make a saw that can't hurt me? And so he had this idea of how do you stop the blade?

Not just how do you keep your fingers away from the blade? How do you stop the blade if you touch it? And so what he did was he ran an electrical circuit through the blade, put a microprocessor on there and put basically a steel brake in there.

And humans conduct electricity. If you touch the blade, you're completing that electric circuit, which fires the microprocessor, which instantly says, fire the brake. It works a little bit like putting a stick in someone's bicycle spokes.

It just stops the wheel from turning. And that's what it does. And he made a thing that if you touch the blade, could stop the blade in a fraction of a second.

So you would get a little nick, maybe, but not anything that a bandaid couldn't handle.

Jordan Harbinger: I remember seeing this maybe on TV or possibly YouTube. I just remember this blade is going and it's making that scary cutting sound that table saws make.

And the guy cuts a piece of wood and like a hot knife through butter, right? Cutting through this two by four or whatever it was. And then he takes his finger and he goes, and the thing goes.

And it makes this very loud pop because I guess these disc brakes or whatever it is.

Alex Davies: Yeah, you're destroying your blade.

Jordan Harbinger: Yes, the blade is totally wrecked.

Yes. But your finger slash hand slash whatever face arm is totally fine. It was incredible.

And I never knew how it worked. So thank you for explaining that. That totally makes sense that there's an electric current going through there.

I'm going to go ahead and guess that every high school shop class bought that as soon as it was affordable to do so. Because the liability on a kid cutting their hand or fingers off in a high school shop class is probably pretty high.

Alex Davies: So when you talk about risk and figuring out how safe does this thing need to be, it's a sliding scale, right?

So this guy, Steve Gass, brought this product to all the big manufacturers. The Consumer Product Safety Commission said, this is great. This should be in the regulations.

Jordan Harbinger: Right.

Alex Davies: But the industry is just like, nope, it's too expensive. And also you own the patent for this thing.

So we're going to be totally beholden to you. So is he altruistic and said, screw it, I'll forfeit the patent? No, he ultimately ended up making his own line of saws, which are called the SawStop saws.

And high school shop classes, he said, are some of his biggest customers because they don't want the liability because they're dealing with stupid high schoolers who maybe don't take off their ties. But it sells worse to just amateur woodworkers. And I actually talked to a product safety expert, a guy who teaches this at an engineering school, and I told him about SawStop.

He says, oh, that sounds great, but I do some woodworking and I wouldn't buy it because of the cost. And that's the product safety expert. But the risk is less.

It was harder to push through because the users are adults.

Jordan Harbinger: I don't know. I kind of want the safest, rapidly spinning, super sharp blade that my fingers are right next to.

It seems like bad math. Oh, man, that's so expensive. How much is it going to cost you, maybe not in dollars, but just cost you in general if you're missing your index finger and your middle finger on your right hand for the rest of your life?

Yep. How much is that going to cost you? Yeah.

Is that worth $300 or whatever? Or not that? I don't even know.

I mean, even if it's a thousand extra dollars, I'm like, no, no, no. And also you have kids in the house. Are they ever going to use it?

Is one of your friends going to come over and use it? Are you worried about their safety at all? Are you going to use it after a couple of beers?

Yeah. You watched the game. Yeah.

And now you want to go finish your woodworking project. And yeah, you've had two beers and you're not exactly as sharp as you were before, but the blade still is. I don't know.

We're bad at doing this math. I think, Alex, like we're bad at doing cost benefit.

Alex Davies: And that's the thing is that engineers have had to learn that we're really absorbed the lesson that people are, can be foolish, so to speak.

People make mistakes and it's not because they're ignorant or stupid or perfidious. It's just because they're people. And so there was an Underwriters Lab engineer who at some point was quoted as saying, why should I design a toaster that can't burn you when you could achieve the same effect by not touching it when it's hot?

True. But guess what? Toasters now, when they're running, don't burn you if you touch them.

People are just going to touch them because they're moving stuff around or they've got their kid in one hand and they're trying to get his toast ready. Mistakes happen. And so the work is really designing against them.

And the testing work specifically is thinking about all the things that a person is going to do with a thing, figuring out that definition of reasonably foreseeable, and then doing your best to design against them.

Jordan Harbinger: The crash dummy thing is, is interesting. We kind of going back to that.

In the beginning, it seemed like all these dummies were kind of men of average height, weight, et cetera. But what about, are we testing products for women and old people and kids and people who are a hundred pounds overweight? I mean, that all would alter data for whether something is safe or not.

Alex Davies: So the crash dummy most commonly used today is called the Hybrid III. GM developed it in the seventies. The basic one represents the, what's called the 50th percentile American male.

So if you take all the men in America in terms of height and weight, he's the middle. He's the perfect average of American height and weight at the time. This is the seventies.

Jordan Harbinger: So it's like 120 pounds and five foot. This is a skinny guy with not much muscle.

Alex Davies: Exactly.

He's like five, eight. Yeah. Is the average.

So one problem already is that it's 2026 and Americans are bigger than they used to be. What year was this?

Jordan Harbinger: 1970 average.

Yeah. Call it 1975. Over the last 50 years, the average size of an American man has changed significantly while the average height has stayed relatively flat.

The average weight has risen substantially. Go figure. Okay.

Well, we could have predicted that. Okay. 1975 average height, five foot nine.

Now we're five foot 8.9. So who knows? That could be like immigration or something. A slight decrease weight was, oh, 1975, 171 pounds.

I'm surprised actually that a five, that's actually a decently fit guy. Now it's 200 pounds. Yeah.

So wow. Okay. It's a 27 pound, whatever, 28 pound increase.

That's pretty significant at the same height.

Alex Davies: It's significant. People move differently in car crashes if they're different weights.

So, okay. So that's one problem. Problem number two, the female dummy is smaller and lighter, but it's not actually made to represent the female body.

Jordan Harbinger: It's literally, it's just a scaled down version of the male dummy. So not to be crude, but my question is, did they put boobs on it? And I know that sounds crude, but the reason is because don't seat belts and things and airbags.

Isn't that going to change if you've got two boobs?

Alex Davies: Yeah. The scale down male dummy much more closely represents a horse jockey than a woman.

Luckily the 2026 American male seems to have boobs. So I guess, I guess maybe it all comes out in the wash, but except we're still using the 1970s dummy.

Jordan Harbinger: Oh, we are?

Oh, geez. Okay. I didn't know.

Alex Davies: They haven't updated it. It's not bigger or taller or different height. It's the 1970s 50th percentile.

And the child dummies are a little better. They're also scaled down versions, but we don't have dummies that represent older people who are much more likely to be hurt in car crashes. There's a dummy that's sometimes used for R&D, but not official certification testing.

That's like a 95th percentile male, a much bigger person. But a lot of Americans are obese or overweight, medically overweight, and they're not accounted for in these test dummies. But they're what we use.

And we know all of these problems that these are the things that tell us whether or not car crashes are going to...

Jordan Harbinger: Are going to be survivable. Right.

A 95th percentile male for people wondering is 6'2", 246. That's a big dude. That's a real big dude.

They do have that big dummy.

Alex Davies: And there is movement toward a new version of a dummy, but it takes forever because it's bureaucracy. And then there's this going back to that issue of workability.

The dummies work because they give you consistent data. If you change the dummy, all of a sudden, you don't know how a 2028 Nissan Sentra compares to the 2025, which is probably something you want to know.

Jordan Harbinger: Interesting.

So I always worry, what is it called? Goodhart's law, right? Once passing the test becomes the target, the test starts becoming less useful.

So if we're just testing crash scenarios where the car is doing well, or like works for the average American male, it's not really a useful, because most people aren't, I mean, average, yes, okay, fine. But most, this is always one of those math concepts that's tricky. Most people aren't average.

And it's like, but it's average, right? But it's the big people plus the small people, not the, what is it? The median or the mode.

It's one of these things I haven't studied since high school. But most people are not that way. That's just the average.

So it could even be the case that only a small number of people are actually that.

Alex Davies: Yeah. There's a really good case for trying to be five foot nine and 170 pounds.

Right. Yeah. You're the thing they're actually testing for.

It's like being sample size with clothing.

Jordan Harbinger: Oh, we don't have your size. Well, that never happens.

They always have my size pretty much, you know, like I'm good. Or it's the size that gets picked over first. I suppose there's that.

Depends on the clothes.

Alex Davies: There's this other issue of designing to the test, right? The car, one of the main crash tests we use is just a frontal crash.

Car hits a barrier straight dead on. And cars are all designed to pass that, to limit the G-force passed on to the people inside the car, as long as they're wearing their seatbelts and their airbags are functioning. But then if you change the conditions of the test, the cars don't always do so well because they're designed to meet that test.

It's like educating a kid to take a standardized test. But what are you missing? You're missing everything outside the exact parameters of that test.

And the proof we have for this comes from the Insurance Institute for Highway Safety. And this is a nonprofit funded by insurance companies that have a direct financial interest in limiting crashes. And they run their own tests that are usually stricter than the official federal tests.

And they give out their own safety picks, which are actually really important selling points for cars. If you get an IIHS top safety pick, it is going to be in every car commercial. So if you hear top safety pick, it means IIHS said this thing passed all of our tests.

But they're tricky because what they did at some point was they said, you know, cars are really good at this direct frontal thing, but we have a lot of evidence that cars don't always hit things dead on. They might hit a telephone pole. You're not always going to hit it like this.

You might hit it like this, sideways to the side. They call that the small overlap test. And when they started running that in the 1990s, a lot of cars failed it.

They were protecting a perfect dead on hit, but you move it two feet over and all of a sudden those crashes get a lot more dangerous.

Jordan Harbinger: Oops, it rips the car in half because of the shearing force or whatever. Yeah.

And then people spill out the back and hit the wall with no car around them.

Alex Davies: The good news is car makers are so invested in getting these top safety picks that they said, okay, we'll engineer the car to meet this test. And then IIHS moved the goalposts again.

They said, we're going further over, right, to the passenger side. We want it on both sides. And that's where testing can not just verify a product, but actually drive a product to be better.

Jordan Harbinger: So the government also tests things, right? There's a chapter where there's a federal tea guy. This is such a weird job.

You've got to wonder how he ended up doing this for a living.

Alex Davies: For about a hundred years, from the 1890s to the 1990s, the United States hired people to taste test every batch of tea that was imported into the United States. And the reasons that created this law were very 1890s reasons.

They were being sent all this crappy tea, tea that had already been used and it was just repackaged.

Jordan Harbinger: And that's so gross. So somebody brewed the tea leaves and then put them in a box and was like, sell these to the Americans.

They don't have any taste.

Alex Davies: Or they were like bad leaves or mixed up and they're like, here you go.

Jordan Harbinger: And Americans were being sold the dregs of tea.

So this is like the weed dealer who gives you oregano and you're like, it's not working. Oh, you just have a tolerance. Yeah.

Alex Davies: Right. And so it was this unique thing. It's the only product, food or drink product that the government has put rules around how it tastes.

Not just is it going to kill you or make you sick, which are how most things are regulated. So it was this weird exception to a law. It was like the Tea Importation Act of 1896.

And the last guy doing this was this guy, Robert H. Dick. And he worked in an office in the Brooklyn Navy Yard and he'd show up every day, look at the tea that was sent to him.

It would be like a small sample from every crate that came into the U.S. He'd pull out a sample and he'd brew it and he had a lazy Susan so he didn't have to get up from chair to chair and he'd taste it. Swirl it around in his mouth, gargle, gargle it and then spit it out into a little cuspidor.

Jordan Harbinger: And he did this for eight hours a day.

Alex Davies: And he did this for eight hours a day. He would taste like 300 cups of tea in a day. This had been going on for like a hundred years and at a certain point it was mostly useless because tea quality just improved.

Jordan Harbinger: I would imagine it's cheap enough now where you don't have to reuse it or send people fake sawdust from a tea factory. This guy's caffeine tolerance must have been off the charts because even if he's spitting it out, he's still absorbing it through the mouth.

Alex Davies: Yeah, you're absorbing some.

And so various presidents were like, this is very stupid. Why are we spending money on this? Like Carter tried to get rid of it.

Nixon tried to get rid of it. But you needed a congressional act to get rid of it because it was a congressional act that mandated the system. And so under Clinton, they finally got rid of it.

But my favorite little side story about this is that at one point Nixon said, I'm going to get rid of this because it's costing money. And a group of UC Berkeley students sent him a letter saying, if you really want to save money, how about we sent fewer troops and munitions to Vietnam and keep this product? And remember, the guy's name is Robert Dick, and they encouraged President Nixon to keep Dick tasting.

Jordan Harbinger: Oh, God. Yep, I see it. Keep Dick tasting.

Contingent Magazine, August 8th, 2021. A U.S. Tea Tasting Commission actually existed, formerly called the Board of Tea Examiners.

Alex Davies: Yep, and they would set standards for how tea should taste, and then Robert Dick knew those standards, and he'd test tea and be like, yep, that meets that standard.

Jordan Harbinger: Wow. Robert H. Dick has spent his entire 56-year government career tasting tea, and you can bet he knows his pekoe from his oolong.

I want to see a photo of this guy smiling because I bet you his teeth are completely brown. The only photos I found of him were he was very serious. He took his work seriously.

Oh, man. And then what does he do? He's like, this is terrible.

I'm not approving it. So they had to stamp it and say, this is good to go. Is that kind of his...

Alex Davies: Yeah, sometimes he'd be like, oh, there's grain in here. Occasionally, he'd also stumble upon slightly radioactive tea.

Jordan Harbinger: Whoa.

Alex Davies: For that stuff, he had a Geiger counter. He wasn't taste testing that, fortunately for him.

Jordan Harbinger: Yeah, radioactive tea.

What do you suppose happened there? I don't even want to think about it. Yeah, no kidding.

Alex Davies: But now they got rid of it. And because it's a crazy way to test stuff, this is called the 100% approach to testing.

Jordan Harbinger: Yes.

Alex Davies: And it's better reserved for like Hermès bags and scarves. Yes. That someone's actually going to look at every single one, luxury goods.

And then often also for very important things like airplane switches. Yes. You know, those often get the 100% approval rate.

Jordan Harbinger: I'm curious, what else is 100% tested? The book has some cool examples. One of them was, there's a, is it called Iberico?

The Spanish hams? They're like $700.

Alex Davies: Yes.

A very fancy ham company has these guys called caladores, who as the hams are curing, they have these little picks made of bone. And they go and they pick a hole and they smell it. And they just know at this stage of curing, this is what it should smell like.

Or this one's actually off, so we're not going to sell it.

Jordan Harbinger: Wow. That's one of those crazy jobs.

It's a tea taster, ham sniffer.

Alex Davies: Le Creuset stuff is 100% inspected.

Jordan Harbinger: I don't know what that is.

Alex Davies: Le Creuset, the very fancy kitchen brand.

Jordan Harbinger: Oh, okay. Okay.

Alex Davies: Where you get a, like a Dutch oven and it's $700 or something.

Jordan Harbinger: And then every, I might say this wrong too, Zildjian cymbal?

Alex Davies: Yeah, Zildjian.

I'm actually not 100% sure on the pronunciation. It's a cymbal maker, like in a drum kit. They've been making these cymbals since Istanbul was still called Constantinople.

Jordan Harbinger: Wow. So this is centuries old. But how do you test a cymbal?

You hit it with a drumstick and if it makes the right sound, you're good.

Alex Davies: You hit it with a drumstick and you have a guy who knows exactly what it should sound like. And he spends his day going.

Jordan Harbinger: Okay. This one's warped. Yeah.

I think they make gongs too, or maybe it's just a giant cymbal. I rented a studio in SF once and I was interviewing Neil deGrasse Tyson. And he came in and he goes, whoa, could I hit this?

And they're like, sure, man. And he wound up and he hit this thing and he goes... And then so the show started with him going, you know, those vibrations are now radiating throughout the entire universe.

And I was like, oh, this is a great way to kick things off. He's a showman. He is a show...

Yeah. And it was kind of funny because it was like 7 a.m. And he had just gotten off a red eye from New York. He looked like he didn't sleep a wink at all, came in and was kind of just like not in the mood to do a show you could tell.

But then he immediately was like, no, I'm not going to... I'm already here. Let's do this.

That's interesting. They strike the cymbal because you would think you could just look at it and go, it's fine. It's in the same design as all the other ones.

Alex Davies: Or there might be some computer test. No.

Jordan Harbinger: Yeah.

Alex Davies: The best way to do this is just to have a guy who knows exactly what it should sound like.

Jordan Harbinger: Yeah. God, what a crazy job that is.

I mean, maybe they're also doing other things. Like maybe they're making it or maybe the... I don't know.

Yeah, I don't know. Huh. So interesting.

So things that aren't 100% tested, what is kind of the acceptable number of defects in certain things? If you come across any products where you're like, wait a minute, they accept a 1% defect rate in this, that's kind of awful.

Alex Davies: Yeah.

So it depends on the product and it depends a little bit on the manufacturer and the industry. And there are all these statistical tables for this of if you want to hit a 5% defect rate, you have to test this many things and you have to look at this many different things. But most regular products you buy that are not life and death, like a shirt or a piece of Ikea furniture are actually designed or the inspection process is set up to let a certain number of defective products through.

And it might be something like 5% can be bad or maybe for clothes, it's like 10%. And then there are different level of defects. So if you're buying Spider-Man pajamas for your kid, right?

If one has Superman on it instead of Spider-Man, that's a fundamental defect. Like that's too bad. Yes.

And that's, you're probably going to reject the entire lot because of that, if it has even one of those. But if, oh, one leg is slightly longer than one on two out of a hundred, that's kind of a minor defect and you're like, okay, that might pass. There are all these little rules around it, but it's actually really changed the way I read product reviews for things, consumer reviews.

Because if someone says like, I bought this thing and the screw hole didn't line up with where the screw had to be, I think that's probably a minor defect. And this person is probably just got one of those 2% things that you knew wouldn't work. And that's how I think about it.

When I get something that's slightly messed up, it's not this product as a whole is bad. I'm thinking, well, there's only a 2% chance or whatever it is. I'm going to have the problem this reviewer had.

And really, I don't know who writes reviews. If you get a piece of Ikea furniture and you're like, well, I put it together. Here it is.

Obviously, people write more negative reviews than positive reviews.

Jordan Harbinger: That's true. This reminds me of when you look at Amazon reviews and something has a negative review and it's like arrived damaged.

And you're like, okay, well, you're not reviewing the product. You're reviewing the shipping mechanism. I don't think they made only damaged books.

Alex Davies: Right. And what are the odds that your UPS guy is going to be less careful, more careful? Like it's, I usually just buy the thing anyway, if it looks good.

Jordan Harbinger: Of course. Well, who gets to decide what's an acceptable defect when the producer is the one who saves the money, but the customer is the one that absorbs the failure. Although I guess you can return it.

And then the seller absorbs the failure.

Alex Davies: You can return it. And there are actuaries doing these equations inside these companies saying, well, you can have this many defects before it's more costly to throw out the lot than it is to deal with angry customers.

So if you have a 50% defect rate, you've got a financial problem on your hands. But if you've got a 2% defect, right, you can probably get away with it. That number is somewhere in there and it's just math.

Jordan Harbinger: You saved a couple of bucks on that table saw. Great. Now count those savings on the fingers you have left.

We'll be right back. This episode is sponsored in part by Pendulum. Ever finish lunch and find yourself reaching for a snack an hour later or feel those afternoon sugar cravings kick in even when you're trying to eat healthy?

Metabolism involves more than calories in and calories out. Your gut plays a role too. Pendulum Metabolic Daily is a probiotic designed to support your body's metabolism, help you feel fuller, longer experience, fewer cravings and make it easier to stay on track with your health goals.

Just one capsule a day with food. Some customers report noticing fewer cravings and more energy in as little as a week, though individual results can vary. Developed by PhD scientists from Harvard, Berkeley and Stanford, Metabolic Daily combines five probiotic strains, including Akkermansia, a gut bacterium studied for its role in gut and metabolic health.

The formula supports healthy blood sugar and metabolic health to help promote steadier energy throughout the day.

Right now, podcast listeners can save up to 48% off their first order at pendulumlife.com/Jordan. That's pendulumlife.com/Jordan to save up to 48% off your first order. Again, that's pendulumlife.com/Jordan to get almost half off today.

All right, I'm at Jordan Harbinger on Twitter and Instagram. You can also connect with me on LinkedIn. Here's the rest of my conversation with Alex Davies.

One practical tip that I gleaned from reading the book that was especially relevant lately was there is an unexpectedly practical piece of consumer advice in here from this food safety lawyer, Denis Stearns. So he buys whole heads of lettuce instead of bags of pre-cut lettuce, which I mean, now with all this cyclosporidia or whatever it's called. Tell me why he does this, because clearly he knows something that we don't.

Alex Davies: He does that because when you buy a bag of spinach, those leaves could be coming from lots and lots of different plants. Lots and lots of different spinach plants, maybe even different farms. And when you increase the number of plants that you're eating, you're increasing the odds that one of those had bad stuff in it.

Eating one solid head of lettuce isn't perfectly safe because it could have cyclospora in it. But if you're eating a salad that comes from 100 different heads of lettuce, your odds are 100% greater. And so it's kind of annoying to deal with the head of lettuce because you've got to cut it, you've got to wash it, you've got to get out the salad spinner, and you've got to dry it.

And it takes up your whole dish rack. And when you're doing dishes, it's a pain, but it's a lot safer. And it's actually advice I've mostly taken to heart.

Jordan Harbinger: It seems like a pretty good idea to do that sort of thing, especially nowadays. Yeah, it's an interesting way to think about it, right? Like the more you add processing or supply chain variables, the more likely there is for something to go wrong.

Whenever you read something like, people are finding screws in their beans. It's like, what is going on here? And then you find, oh, yeah, well, it's only a certain kind of bean.

And it's because they have to buy them from all over the place. And they usually come from these developing countries. And they use this kind of machine.

And this machine is built in China with all these loose screws that require a lot of maintenance. So the odds of getting a screw in this brand of this kind of bean is like 1,000 times greater than finding a screw in any other kind of bean. It's always some sort of weird case study looking thing when you dive into it.

Alex Davies: And it's usually probably just going to be one can of beans.

Jordan Harbinger: Yeah.

Alex Davies: And you find a screw in it.

Like I once bought a can of Coke and it was empty. Yeah, that's weird. It was sealed.

It was empty, which is just like a fluke thing. And I'm just like, okay, well, that cost me a dollar and didn't stop me buying Coke. Fundamentally, because I'm like, this is never going to happen again.

Jordan Harbinger: Right. Yeah. The odds are super slim.

You could also, I feel like you call the number on the back of the package and they send you a coupon for a free 12 pack or 24 pack. And you're like, wow, I made money on this. Tell me about the Golden Gate Bridge.

This is a little different than product testing, but this is amazing because it's such a massive project. You can't sort of, all right, well, in the next one, we're going to fix these issues. You've got this 80 year old bridge and it suddenly becomes a musical instrument.

Alex Davies: So in the 2000s, the people who run the Golden Gate Bridge, they wanted to add netting under the bridge to prevent, or I suppose, mitigate suicidal jumps, which are unfortunately common issue on the bridge. And they wanted to sling these big nets under. So if you jump, you don't die and you can get help instead.

The thing is those nets were heavy. They just have to be heavy because they have to be strong. They have to support a person who's jumping from a given height.

And they thought about it. They said, okay, this is going to add weight to the bridge. So let's make sure that the bridge, which is a structure that's built, that we built in the 1930s can support that extra weight, that it's not going to create new issues.

And this is what I call dynamics testing, where you're looking at how one thing affects another thing in the environment. And what they ended up doing was adding weight and we're adding a wind load. So when the wind blows, there's going to be more force on this because the nets are going to catch some of that wind, even if they're nets.

And the way they figured it out was not to make the bridge stronger, but to decrease the wind load on the pedestrian railing. So the pedestrian railing originally had slats that looked like the slats on a picket fence. They faced the wind.

So they picked up a lot of wind load and they said, well, we're going to offset the net added wind load by just basically putting in new slats that are turned this way.

Jordan Harbinger: The opposite way. Okay.

Alex Davies: The opposite way. And so they're going to catch a lot less wind and they kept the spacing more or less the same. And they brought this thing into a wind tunnel. They brought a scale model, which kind of makes sense because you're dealing with the Golden Gate Bridge.

So it's like a one eighth or something or one 30th scale model with little nets on it. And the things turned and they said, okay, the wind load is really good. This bridge is as strong as it needs to be still.

And now we've got the added benefit of the safety nets. So what they didn't find out in that scaled down testing is how changing the slats would change the acoustics of the bridge. And they missed it because who's thinking about the acoustics of a bridge.

And because if you actually want to test for the acoustics, it's very hard to do that with a scale model. You actually kind of need the full thing. Otherwise it's very complicated and you can do it, but it involves a lot of math, like math with Greek numbers and this thing called the Reynolds number.

And it's very complex and you need specialists who can do it. Anyway, the bridge started humming. They turned the slats and what they essentially created was the world's largest harmonica.

So you'd get these big winds coming off the Pacific ocean, going through the bridge. And all of a sudden you heard this noise across the bay. Like in Oakland, you'd hear like, like that noise you get in a horror movie.

Jordan Harbinger: Oh, that's awful. Imagine living near that.

Alex Davies: And the people were like, this is terrible.

And it would go on like for hours.

Jordan Harbinger: Sure.

Alex Davies: And different winds would create different like different humming noises.

The way you, if you change your mouth angle on a harmonica, it changes the sound. And this was a real issue for people, mostly because it was really annoying. It didn't cause any safety issues or anything like that.

Although the other thing, and I don't cover this in the book, but it's another real issue was they did this. A lot of people bike across the bridge. It's a very popular tourist thing.

It's not really a way to get from one place to another. Yeah. I've done it.

I lived in San Francisco at a certain point. A friend comes to visit.

Jordan Harbinger: Exactly.

Alex Davies: Oh, we'll bike, we'll rent bikes. We'll go across the bridge.

Jordan Harbinger: Exactly.

Alex Davies: When they turn the slats, you get a lot more wind. And that wind coming off the Pacific ocean into the Golden Gate Strait is serious.

Jordan Harbinger: Yeah.

Alex Davies: And so this became a real issue for cyclists.

Jordan Harbinger: I'm curious. What about misuse that you kind of know is coming?

AirTags is a really good example. Apple created AirTags to find lost objects. The first ad that they had for it, I thought, oh, so they're just going to allow people to buy tracking devices for other people using these things.

It's really good at locating objects. I use them all the time. I own a dozen of these things.

They're also really good at locating people who don't want to be located. They've done some mitigation of that, but Apple had to know on day one that stalking was possible and they built some countermeasures, but I don't think they were enough because I've seen a million articles about this.

Alex Davies: Yeah.

And I call this idea intention testing, which is not foolproofing. I think of as mostly accidental misuse. You touch the toaster when you didn't know it was hot.

Intention is how is someone actually going to use this? And if the product works, is it producing the thing you want it to do? So Apple made AirTags so that I could find my keys easily, but people use them to find their ex-girlfriend a lot.

And people knew this from the beginning and Apple knew this from the beginning, even as they were developing it, because these things are tiny, they're totally inconspicuous, and people would slip them under the seat of a car, into the seat of a car of their ex-partner or their wife, or it was a very abusive practice, or if you don't have custody, if you're a kid, but you want to keep track, you put it in their backpack and they're not going to notice.

And Apple did build some countermeasures in from the beginning where if you were moving around and an AirTag was kind of matching your location, after a while, it would say, Hey, like there's an AirTag with you that isn't associated with your account.

Jordan Harbinger: This happens to me all the time. Because my kids will go for a walk with me and I guess their AirTags are on my wife's Find My, and it's shared with me.

So I don't really get why it's saying that, but, or, or her AirPods will be in the car and I'm driving the car and say, Hey man, there's an AirTag moving with you, but it's not, it's not that specific. I don't think, I don't know if they can let me find it. It sort of begs the question, right?

Like what's a company's obligation here? You've foreseen a malicious use. Do you just warn against it?

Or do you have to make the abuse more difficult? Even if it makes the legitimate product use worse, you could make them less accurate, but then it's like, well, crap, these aren't as useful anymore.

Alex Davies: It's one where it's really hard to stop the misuse because how are you going to stop abusive people or obsessed people or stalkers?

Just like you can't stop a fool. You can't stop those people from doing what they're going to do with your product. So you have to focus on the product and you have to think through, okay, if someone is using this this way, how do I make that less effective?

And so also Apple got hit with a class action lawsuit of people who had been tracked unknowingly and unwillingly by AirTags. And the last I checked that was still kind of slowly moving its way through the courts. But in the meantime, Apple has taken various steps to beef up those things.

To shorten the window in which you get that warning, to make it a little more aggressive. They worked with Android so that if you're not an iPhone user, but an AirTag has been slipped into your purse, your Android is able to tell you that too. So they did take steps.

And I have noticed when I travel with my husband, I get those warnings relatively frequently. And like we were on vacation this summer. I was like, stop telling me we're on vacation.

We're together all the time.

Jordan Harbinger: Yeah, that's funny. Yeah.

You should be able to say what AirTag is it? It should tell you who it is. Oh, it's linked to Jen Liao.

Okay. Well, that's my wife. You can ignore this forever now.

Alex Davies: And maybe that's the next step. But then you get into privacy issues.

Jordan Harbinger: And so that's true.

This reminds me of DEF CON, the hacker conference. A company will go there and say, hey, our security people tested this, but you guys are kind of like bad guy adjacent, mostly good guys. But what if we let you hack ATMs and we weren't going to call the FBI as a result?

In fact, if you hack into this ATM that we're going to put in a hotel conference room and give you unlimited access to and you are successful, we'll give you a hundred grand if you can make it do certain things. And we'll give you 10 if you can make it do other things. And this is brilliant, right?

Because I think the company pays a little bit to sponsor that and go there. And then they basically just bring a couple of ATMs and they tell some of the world's most brilliant hackers, hey, you can plug things into the USB on the back. We're going to pry that open for you.

Normally you wouldn't have access to that, but why not? Theoretically, you could find it if you had unlimited time. Go for it, man.

This is, there's no money in there, or maybe there is and you get to keep it if you open it. I don't know. But it was just like, it's a really great way to have brilliant people do product testing.

They're just a bunch of Kobuk the Destroyer type of kids.

Alex Davies: Yeah, and that's the most effective way of testing, right? Is often it's cranking up the similarity idea of, well, who's likely going to hack into an ATM?

Someone who has a good idea of how to hack into an ATM. The dumb person is just going to steal the ATM and I guess go at it with an axe and you can build against that. But you need to build against the smarter person too.

And DEF CON is a great place to find those smarter people. Which is why a couple years ago, DEF CON sponsored a thing to get people to jailbreak LLMs.

Jordan Harbinger: Yeah, AI.

Alex Davies: AI chatbots.

With AI, one of the best ways we have currently to test it is to red team it. Red team testing is basically that idea that we just talked about with hackers and ATMs. You just try to attack something in a slightly controlled environment. And so DEF CON sponsored this test where they put out a couple hundred computers, each had a randomly assigned AI chatbot, ChatGPT, Claude, Gemini, these different products.

It was open to all DEF CON attendees. You were given a list of challenges. All of which, theoretically, the model should have been designed to not let you do.

Like, get it to get a math problem wrong. Get it to give you bad financial advice. Get it to confirm that Qatar is the world's largest producer of iron ore.

Jordan Harbinger: Yes.

Alex Davies: When it is nowhere near the largest producer. And however many of these challenges you can break, whoever ticks off the most challenges gets a big prize, which I think was an NVIDIA GPU.

Jordan Harbinger: Which is great because these are people who love that kind of thing.

Alex Davies: Yeah, exactly. And they found, I think they got a thousand attendees about that.

And they ticked off a lot of these boxes.

Jordan Harbinger: And then of course the agreement is, we get to see everything you did. And then reverse engineer it.

And maybe you can even make suggestions on what happened there. If they find something really serious, they'll probably call you. Because DEF CON is cool.

When I went, this is a long time ago, maybe it's bigger now. I don't know, it's like 16,000 people. And it's generally a lot of the best hackers in the world.

So you'll meet people there and go to a dinner. And you're like, what are you doing? They're like, oh, uh, I do some kind of security for some big companies.

And then later on, someone's like, what his job is, is to break into Amazon warehouses and steal things. But he can't tell you who he works for or what he does. And you're like, whoa, what's that like?

Even I've done some red team gigs where it's like you get a letter from essentially corporate and security. And you get to go in with your high-vis vest on and go, yeah, I need to inspect this such and such, whatever thing. And they're like, we're not just going to let you in.

You're not on the list. And you go, look, it's your ass on the line if I can't inspect the server room right now. And they're like, oh, God, OK, fine.

And then you're like, well, I'm in the server room. And it took 80 minutes. And all of the security guys came down.

And then they left because I was taking too long. And I'm standing alone in the server room. And if they call the police, you've got this letter that says, call this guy.

And he'll tell you I'm actually supposed to be here and not actually unplugging things. It's quite interesting. And it's way cheaper than having somebody actually go into an Amazon warehouse and set the server rooms on fire, for example.

Alex Davies: The trick with what's happening with AI, and I'm just talking about models that you chat to. We can talk about the Hugging Faces, the agents.

Sure. How much crazier and harder that makes things to test. The thing is, it's not just DEF CON engineers who are going to use LLMs the wrong way.

It's everybody who uses one. And this testing gets exponentially harder when you make the pool of people who are going to use this almost infinite. And the ways in which they're going to use it, right?

There are only so many things you can do with a screwdriver. Yes, you can stick it in here. You can pry open a can of paint.

You can turn screws. Like you can hit something. You can come up with all the things you can do with it.

You can't come up with everything that someone's going to ask a chatbot.

Jordan Harbinger: Exactly.

Alex Davies: And that testing just gets harder.

And like, it's just trying to get as far as we can.

Jordan Harbinger: AI safety is a whole different episode. I mean, recently I did a show on synthetic biology and how AI might make it possible for a high schooler or a college student to make a deadly pandemic.

And as I was creating the notes with AI, it was like, I can't do this because this is sketchy and you can apply for trusted access, but we're going to limit you. And then I asked it something about a video game or whatever. And it was like, we're giving you all of your requests to extra scrutiny.

This is going to take longer because of stuff you've asked before. But what I felt was interesting was I made a new chat. I did the exact same thing I was trying to do in that last chat in two separate new chats.

And it was like, no problem. So I was like, ugh, guys, you're giving me extra scrutiny. And all I did was click new chat, do one half of it and then click new chat.

And I did the other half and you did it instantly. Like, okay, this thing needs a little bit of work. Come on.

Alex Davies: This stuff is really hard to test. And I do take heart though. I wonder now, is AI the thing that is beyond the arm of testing?

Because testing is an arms race. You build an airplane, you need a way to test it. You build a new safe, you need a new way to test whether or not someone can get in.

Smarter bears, better bear canisters. So testing has historically always caught up. And I found this really encouraging quote from an Underwriters Lab engineer from the twenties talking about electricity in the home.

Because all of a sudden people had like electric irons instead of a hunk of iron. You put in the fire to heat stuff up and people were afraid of risks and fires. And Underwriters Lab was there to actually do testing of these things.

And that's where testing really became a business for the first time. And he said, we just have to get used to it. We had to get used to hydraulic power.

We had to get used to steam power. We had to get used to faster trains on tracks and figure out how to test each of those. And electricity is just the next frontier.

And look at now, like I am not worried that my iron is going to set my house on fire, even if I leave it plugged in. But I'm terrified of AI. And so I take heart that I think people hopefully are working on it and hopefully will catch up or get closer.

Jordan Harbinger: You've spent years immersed in exploding airplanes, defective products, catastrophic surprises, and every imaginable way human beings get things wrong. So after all of that, what makes you more optimistic about our ability to build things that we can actually trust?

Alex Davies: Kids toys.

Jordan Harbinger: Really? Okay, say more.

Alex Davies: Kids toys.

I read a lot about how those things are tested. There are very strict rules because they know that kids are going to put things in their mouths or hurt themselves with them. And when you make a new doll or something, it actually has to go through a list of testing labs where it's yanked with the strength of a toddler plus.

Actually, it's probably yanked with the strength of an adult to make sure a toddler can't pull it apart. It's lit on fire or put to the flames to see whether or not it'll catch on fire. All these little things are going to swing it and poke it and look for things that could come off and a kid could swallow.

And I have an 11-month baby at home. And for a while, her favorite toy was our silicone pastry brush because she would chew it. And when she didn't have teeth, I was like, this is fine.

She can't do it. But then she got teeth, unfortunately, and she was chewing on it. I'm thinking this thing isn't made for this.

It probably wasn't tested for this. But when she goes in her playpen and pulls up the mat we've put down and starts chewing on it, I think this thing was tested for her.

Jordan Harbinger: Sure.

Alex Davies: And so I'm like, that's fine. She's not going to pull that off. And I even tried to yank it as hard as I could to see whether or not she'd pull it off.

I do think testing has solved a lot of problems. And that makes me optimistic. And it has caught up to a lot of things that originally outpaced it.

And that makes me more optimistic.

Jordan Harbinger: Alex Davies, thank you so much. I appreciate you doing this when you have a cold, because I know how tired I am after this.

Yeah, I need a nice hot tea that's maybe been tasted and spat out by what was his name? Richard Dick? Dick Dick?

Robert Dick. He was Bob Dick. Yeah, Bob Dick.

Thanks to Bob Dick for tasting all of our teas and now we don't have to worry about it. Thanks to you for coming all the way to New York with a cold to do this show, attempt number two. I really appreciate it.

Alex Davies: Yeah, thanks for having me.

Jordan Harbinger: How could a war halfway around the world affect what you pay or what you can buy? Here's a preview with Peter Zeihan on what happens when the countries making our everyday essentials can't keep delivering.

Peter Zeihan: Since 1989, there's really only been one large military industrial complex in the world that's been American. And most of our allies buy our equipment. And what we're discovering now is a lot of these new technologies are really effective at destroying some of these types of equipment in certain environments.

The technology is evolving so quickly, the rules of the game are changing, the goalposts are moving, and every state in the Persian Gulf and every front and second line state in NATO against the Russians has already broken ground on mass manufacturing facilities to make Ukrainian drones at scale. So it's entirely possible that two years from now, every ally we have is not going to be buying meaningful military equipment from the United States because it's just not applicable anymore and we won't deliver anymore. We are seeing deglobalization accelerating at the same time the United States is deindustrializing.

If anything, we need to be sprinting in the opposite direction. We're no longer building ourselves out for what is coming. We're just kind of waiting for the world to happen to us now.

To keep it going, you need the United States to maintain the military structure that allows global trade to happen. It never occurred to me that we could have an administration so out of touch with reality that we would do this on purpose. This conflict is going to go down in history as the moment where the United States lost the ability to influence the Eastern Hemisphere.

Jordan Harbinger: To hear how working with Canada and Mexico could help America weather the chaos and what's standing in the way, check out episode 1370 of The Jordan Harbinger Show with Peter Zeihan. My big takeaway? Good design assumes people are going to screw up.

A warning label doesn't make you less tired and knowing better doesn't stop your hand from slipping. Ask yourself, what mistake is this setup counting on me never making? Then change the setup so the mistake is harder to make or just less costly when it happens.

I'll just be more careful is a terrible safety feature. All things Alex Davies will be in the show notes on the website. Advertisers, deals, discounts, and ways to support the show all at jordanharbinger.com/deals. Please consider supporting those who support the show. Don't forget about Six-Minute Networking as well over at sixminutenetworking.com, always free, just like this show, which is created in association with PodcastOne. My team is Jen Harbinger, Jase Sanderson, Robert Fogarty, Tadas Sidlauskas, Ian Baird, and Gabriel Mizrahi.

Remember, we rise by lifting others. The fee for the show is you share it with friends when you find something useful or interesting. In fact, the greatest compliment you can give us is to share the show with those you care about.

If you know somebody interested in product testing or other nerdy engineer brain type stuff, definitely share this episode with them. In the meantime, I hope you apply what you hear on the show so you can live what you learn and we'll see you next time.