1 00:00:12,139 --> 00:00:13,139 [Music] 2 00:00:13,269 --> 00:00:16,570 MC: so have you ever called IT only to be told 3 00:00:16,570 --> 00:00:18,570 have you tried turning it off and on again? 4 00:00:23,200 --> 00:00:28,460 Today we will be talking about pilots usually pilots are called to pilot 5 00:00:28,460 --> 00:00:34,640 in command and we expect a pilot to be just that in command but today's pilots 6 00:00:34,640 --> 00:00:39,829 are turning more and more into computer operators and have less and less actual 7 00:00:39,829 --> 00:00:47,860 hands-on flying ability so now imagine you are the pilot flying a gigantic computer 8 00:00:47,860 --> 00:00:55,070 in 30,000 feet height with 200 souls behind you only to be told by IT have you 9 00:00:55,070 --> 00:01:03,130 tried turning it off and on again so I would like to welcome Bernd Sieker. who is 10 00:01:03,130 --> 00:01:07,950 a systems engineer and an aviation accident analyst he specialized in reverse 11 00:01:07,950 --> 00:01:12,950 engineering and he's developing formal methods to development of safety critical 12 00:01:12,950 --> 00:01:18,970 systems and he will enlighten us about problems in aviation automation because 13 00:01:18,970 --> 00:01:24,720 apparently every pilot has uttered the words what's it doing now 14 00:01:24,720 --> 00:01:32,549 [Applause] BS: yes thank you thank you very much yes 15 00:01:32,549 --> 00:01:37,960 but first I'd like to learn a bit about the audience so how many of you here in 16 00:01:37,960 --> 00:01:46,420 the hall today are pilots Oh quite a few so commercial pilots? far fewer. ATP 17 00:01:46,420 --> 00:01:54,429 anyone? yeah there's one I heard one but I can't see. okay so some of you will know 18 00:01:54,429 --> 00:02:01,859 about some of the stuff I hope there's a bit new stuff for everyone let's get right 19 00:02:01,859 --> 00:02:09,490 into it what the announcer said was a bit of nice folklore it's not completely true 20 00:02:09,490 --> 00:02:15,009 but there's a little bit truth to it what I'm going to talk about is automation in 21 00:02:15,009 --> 00:02:19,010 the aircraft and the idea is often as he said that it's just a computer and the 22 00:02:19,010 --> 00:02:22,650 pilot doesn't have to do anything that's one saying that in modern airplanes there 23 00:02:22,650 --> 00:02:29,870 will only be one pilot and a dog and the pilot is there to watch what the the 24 00:02:29,870 --> 00:02:33,290 pilots there to feed the dog and the dog is there to bite the pilot if he touches 25 00:02:33,290 --> 00:02:42,099 anything so that's not quite yet how it is I talk a very little bit about the 26 00:02:42,099 --> 00:02:48,860 analysis method that we use to analyze accidents not only in aviation but mostly 27 00:02:48,860 --> 00:02:53,099 and then I'll tell you a short tale of two throttles or two thrust levers as they are 28 00:02:53,099 --> 00:02:58,950 sometimes called and also talk about human pilots how they cope with failures or 29 00:02:58,950 --> 00:03:04,239 don't as the case may be and I haven't seen a lot of other talks here about self- 30 00:03:04,239 --> 00:03:08,260 driving cars although they are now becoming a very big thing so I'll touch 31 00:03:08,260 --> 00:03:14,220 that briefly and have a tentative conclusion. I can't see very fine to the 32 00:03:14,220 --> 00:03:22,940 future so I'm not sure if I'm right about that. So what is automation in airplanes? 33 00:03:22,940 --> 00:03:27,370 the most obvious thing is that automated flight controls on every airliner and on 34 00:03:27,370 --> 00:03:32,810 many small airplanes these days. there used to be a requirement for a simple 35 00:03:32,810 --> 00:03:36,739 autopilot even on small private single- engine airplanes if you want to fly under 36 00:03:36,739 --> 00:03:43,180 instrument flight rules. That has been relaxed somewhat now but many small planes 37 00:03:43,180 --> 00:03:48,660 still have them. so there are three levels of fat controls there the first one is 38 00:03:48,660 --> 00:03:54,670 manual flight where the pilot moves flight controls and the airplane does what it's 39 00:03:54,670 --> 00:03:58,400 told then there's the simple autopilot where the pilot just 40 00:03:58,400 --> 00:04:04,299 sets airspeed altitude climb rate stuff like that and there are managed modes now 41 00:04:04,299 --> 00:04:07,640 where there's a more sophisticated computer which has knowledge about the 42 00:04:07,640 --> 00:04:12,659 whole flight with waypoints and altitudes and there are other automated systems not 43 00:04:12,659 --> 00:04:18,750 only the flight controls spoilers on the ground have to extend to help slowing down 44 00:04:18,750 --> 00:04:25,080 the aircraft the high-lift devices are automated radios maybe Auto-tuning there's 45 00:04:25,080 --> 00:04:30,039 the computer that controls the engines full authority digital engine control. 46 00:04:30,039 --> 00:04:33,840 There are things like cabin pressurization and many other small subsystems are 47 00:04:33,840 --> 00:04:42,160 automated as they are in cars these days. So what is automation not? It's not yet 48 00:04:42,160 --> 00:04:49,060 except for very few specialized drones a self flying aircraft the pilot in command 49 00:04:49,060 --> 00:04:53,010 still is in command at all times you can turn off the automation you can have to 50 00:04:53,010 --> 00:04:58,220 fly the aircraft at any time if you wants to and barring any serious errors which 51 00:04:58,220 --> 00:05:05,490 are extremely rare in commercial aircraft the airplane does what it's told. the 52 00:05:05,490 --> 00:05:09,780 pilots the the autopilot really doesn't have any decision capabilities except at 53 00:05:09,780 --> 00:05:15,190 the very lowest level deciding on a bank angle to make the right turn and things 54 00:05:15,190 --> 00:05:22,010 like that it is also not a panacea for any errors that the pilot can make now you can 55 00:05:22,010 --> 00:05:25,800 still fly a highly computerized modern aircraft into the side of the mountain if 56 00:05:25,800 --> 00:05:32,490 you want to. so some military aircraft actually have systems that will prevent 57 00:05:32,490 --> 00:05:36,080 you from flying into a mountain if they're active or if you have passed out but 58 00:05:36,080 --> 00:05:40,780 airliners don't at the time and of course the pilot in command still bears the 59 00:05:40,780 --> 00:05:46,650 ultimate responsibility for the safe conduct of the flight. so as I said 60 00:05:46,650 --> 00:05:49,730 briefly manual flying is just stick and rudder you move the stick and you move 61 00:05:49,730 --> 00:05:55,979 your rudder pedals and the airplane moves the control surfaces mechanically on small 62 00:05:55,979 --> 00:06:00,690 airplanes hydrolically assisted or even computer-assisted on some airliners on 63 00:06:00,690 --> 00:06:05,919 most modern airliners called fly-by-wire you may have heard about that then the 64 00:06:05,919 --> 00:06:10,560 simple autopilot modes where you directly select the heading and the 65 00:06:10,560 --> 00:06:15,479 airplane flies in that heading and managed modes as I said before where you have a 66 00:06:15,479 --> 00:06:21,710 sophisticated flight management system which then in turn sets headings and climb 67 00:06:21,710 --> 00:06:26,810 rates and things like that on the autopilot proper. they are not super 68 00:06:26,810 --> 00:06:33,880 reliable they can be thrown off by many things and mostly they they turn off when 69 00:06:33,880 --> 00:06:38,930 there's any small error in any of the small subsystems, any of the various input 70 00:06:38,930 --> 00:06:44,009 values that you get air speed altitude engine power anything if any of those have 71 00:06:44,009 --> 00:06:49,789 invalid readings it'll turn off and the pilots have to assume command in that 72 00:06:49,789 --> 00:06:56,800 case. they cannot handle basically anything unexpected most air sensors are 73 00:06:56,800 --> 00:07:01,800 there then threefold so if only one of them disagrees the other two are usually 74 00:07:01,800 --> 00:07:07,520 taken as valid but if they all three disagree then the system just says I don't 75 00:07:07,520 --> 00:07:13,659 know what's true anymore what speed is and all the automatics drop out and most of 76 00:07:13,659 --> 00:07:17,430 the computer assisted manual flying also is turned off in that case. 77 00:07:20,140 --> 00:07:21,800 so this is very briefly the method 78 00:07:21,800 --> 00:07:25,389 that we have developed at the University of Bielefeld 79 00:07:25,389 --> 00:07:30,900 and the professor lepkin for analyzing accidents called why because analysis it 80 00:07:30,900 --> 00:07:36,081 uses a formal notion of causality called the counterfactual test and then you can 81 00:07:36,081 --> 00:07:42,940 make a very nice graph for accidents they're usually bigger than that but it's 82 00:07:42,940 --> 00:07:47,440 more or less objective criterion for causality and then different people with 83 00:07:47,440 --> 00:07:52,150 some experience in the domain make why because graphs of an accident they usually 84 00:07:52,150 --> 00:07:53,370 are very similar to each other 85 00:07:56,630 --> 00:08:00,480 so there's a lot of automation on modern airplanes 86 00:08:01,380 --> 00:08:08,990 and it's quite hard to get it right and one of the reasons is that unlike for many 87 00:08:08,990 --> 00:08:15,159 situations in cars and rail vehicles there is no default safe state, you can't just 88 00:08:15,159 --> 00:08:19,910 turn everything off and stop by the roadside so we always have to decide the 89 00:08:19,910 --> 00:08:26,599 engineers always have to plan for many eventualities what can happen in the air 90 00:08:26,599 --> 00:08:33,209 and decide what given a certain set of circumstances is the safest state for the 91 00:08:33,209 --> 00:08:37,679 airplane to be in and that is not always unambiguous and it's a very hard decision 92 00:08:37,679 --> 00:08:42,190 tomake and sometimes they get it wrong and 93 00:08:42,190 --> 00:08:48,930 sometimes you just get into that situation where in most cases the set of values the 94 00:08:48,930 --> 00:08:56,019 set of measured values that the system gets when most circumstance is one set of 95 00:08:56,019 --> 00:08:59,960 decisions is the correct one and you get into that situation where the computers 96 00:08:59,960 --> 00:09:05,070 get the same inputs and that decision is the wrong one and that may still lead to 97 00:09:05,070 --> 00:09:15,170 an accident. those are very few and very rare but these these things can happen. so 98 00:09:15,170 --> 00:09:19,000 a few of the decisions that the engineers have to take when designing the automation 99 00:09:19,000 --> 00:09:25,279 in airplanes is what to do if things fail if certain individual things fail if a 100 00:09:25,279 --> 00:09:32,899 combination of things fail little motors little engines sensors fail some actuators 101 00:09:32,899 --> 00:09:38,390 fail a hydraulic system fails anything like that what you do in that case with 102 00:09:38,390 --> 00:09:45,190 the remaining systems and what to tell the pilots? well naively you might assume the 103 00:09:45,190 --> 00:09:49,180 pilot wants to know about everything that is broken every little valve every little 104 00:09:49,180 --> 00:09:56,370 system that is broken on the airplane but if a lot goes wrong at the same time then 105 00:09:56,370 --> 00:10:01,170 the decision has to be taken which of these things that have gone wrong are the 106 00:10:01,170 --> 00:10:06,950 most important for the flight crew to know and that's not trivial at all and it can 107 00:10:06,950 --> 00:10:15,220 very easily lead to to sensory saturation of the pilots so they don't know what is 108 00:10:15,220 --> 00:10:20,500 what anymore because from all sides alarms are blaring there are lots and lots of 109 00:10:20,500 --> 00:10:28,130 displays that they have to watch and so certain error messages are suppressed in 110 00:10:28,130 --> 00:10:33,230 certain states of flight certain stages of the flight so as not to overwhelm the 111 00:10:33,230 --> 00:10:38,300 pilot. and some things that may be essential to have on the ground some 112 00:10:38,300 --> 00:10:43,510 functions for example the wing spoilers those are the big the big flaps on the top 113 00:10:43,510 --> 00:10:49,490 of the wings that come up after touchdown are important to have on landing to dump 114 00:10:49,490 --> 00:10:54,570 the lift so the airplane doesn't jump up again. because it is a touchdown still at 115 00:10:54,570 --> 00:10:58,410 the speed at which it could fly at least for airliners, for small airplanes it's a 116 00:10:58,410 --> 00:11:03,040 bit different but airliners are safely above the very lowest 117 00:11:03,040 --> 00:11:07,160 speed they can go when they touchdown so they need to have some means to make sure 118 00:11:07,160 --> 00:11:11,959 they don't jump up again they still do sometimes but not very often but the 119 00:11:11,959 --> 00:11:15,990 spoilers destroy most of the lift so deploying them in the air close to the 120 00:11:15,990 --> 00:11:24,340 ground is extremely dangerous so the computer has to be absolutely certain so 121 00:11:24,340 --> 00:11:28,960 to speak to know that the aircraft is on the ground when it gives the command to 122 00:11:28,960 --> 00:11:34,770 deploy the ground spoilers if it does that a few seconds too early when the airplane 123 00:11:34,770 --> 00:11:40,830 is still a hundred meters up above the ground that will likely be a fatal accident. 124 00:11:47,590 --> 00:11:51,760 so in most at least in most jet airliners not in all propeller-driven but 125 00:11:51,760 --> 00:11:56,709 in all almost all jet airliners there's an automatic thrust management so the 126 00:11:56,709 --> 00:12:01,420 computer does not only control where the nose of the airplane points but also how 127 00:12:01,420 --> 00:12:06,550 much power the engines produce and there are two different one might call them 128 00:12:06,550 --> 00:12:12,420 philosophies between the two major air framers and Boeing and most others to use 129 00:12:12,420 --> 00:12:19,320 back driven throttles so the computer sets the thrust and moves the thrust levers to 130 00:12:19,320 --> 00:12:25,889 match the commanded thrust position and Airbus has a different system where the 131 00:12:25,889 --> 00:12:29,410 thrust levers remain in one position throughout the entire flight basically 132 00:12:29,410 --> 00:12:36,190 after take off when thrust is reduced for the main climb and cruise and descend 133 00:12:36,190 --> 00:12:41,810 and everything they remain in one position and the computer tells the engines 134 00:12:41,810 --> 00:12:46,910 directly which thrust to produce. and there are there's an argument which one of 135 00:12:46,910 --> 00:12:53,300 the systems is better but I'll show you accidents three accidents in which the 136 00:12:53,300 --> 00:13:00,910 thrust system the throttle system played a role. so the first one has a little video 137 00:13:03,610 --> 00:13:07,399 you will see I think there are two different camera perspectives you will see 138 00:13:07,399 --> 00:13:12,680 two airplanes landing of the same class they are small airliners two hundred 139 00:13:12,680 --> 00:13:19,449 people something like that 150 to 200 now landing and the first one is a normal landing 140 00:13:22,609 --> 00:13:25,399 so it's already pretty slow takes its time 141 00:13:28,759 --> 00:13:30,549 and the next one is the accident flight. 142 00:13:31,869 --> 00:13:35,280 it's on the same day it's only minutes apart so on the same Airport 143 00:13:36,630 --> 00:13:40,839 and you can see that one is slowed down and the one other one is still going very fast 144 00:13:45,959 --> 00:13:47,200 so there's the first one 145 00:13:50,930 --> 00:13:52,240 and that's the second one 146 00:13:52,240 --> 00:13:58,610 and as you can imagine that didn't end well 147 00:13:59,350 --> 00:14:03,160 it was one of the worst aviation accidents maybe still 148 00:14:03,160 --> 00:14:06,840 the worst today in Brazil where 200 people died 149 00:14:08,370 --> 00:14:11,560 and as you can see this is a 150 00:14:11,560 --> 00:14:16,570 transcript of of the flight data recorder the digital flight data recorder and the 151 00:14:16,570 --> 00:14:23,600 first two lines are the interesting ones that says TLA that is thrust lever angle 152 00:14:23,600 --> 00:14:27,380 and normally what happens on landing just before touchdown the pilot pulls both 153 00:14:27,380 --> 00:14:32,889 thrust wheels to idle now the engine thrust goes down to to idle and then it 154 00:14:32,889 --> 00:14:37,790 touches down engages reverse thrust spoilers brakes everything to slow down 155 00:14:37,790 --> 00:14:42,970 and what happened in this case is that the pilot only moved one of the thrust to idle 156 00:14:42,970 --> 00:14:48,199 and left the other there put the one thrust lever in reverse but not the other 157 00:14:48,199 --> 00:14:54,220 and that led to the computer getting conflicting information about whether the 158 00:14:54,220 --> 00:14:58,970 pilots actually wanted to land or not so it didn't deploy the automatic wheel 159 00:14:58,970 --> 00:15:07,000 brakes it didn't deploy the spoilers and reverse thrust only on one engine so that 160 00:15:07,000 --> 00:15:14,829 went pretty badly and some people said well with tactile feedback from a thrust 161 00:15:14,829 --> 00:15:19,029 levers if the pilots have been used to that they would have noticed earlier and 162 00:15:19,029 --> 00:15:24,380 we can't really be sure because the pilots also died in the accident but there were 163 00:15:24,380 --> 00:15:28,930 some people who made a case that moving thrust levers would have been a lot better 164 00:15:28,930 --> 00:15:29,870 in this case 165 00:15:31,310 --> 00:15:33,330 so is that always better? 166 00:15:36,020 --> 00:15:38,140 here's another throttle related accident 167 00:15:39,560 --> 00:15:46,810 in this time it was a Boeing Boeing 737 at Amsterdam Schiphol Airport there was a 168 00:15:46,810 --> 00:15:50,990 small technical malfunction would call which caused the computers to think the 169 00:15:50,990 --> 00:15:55,470 airplane was actually eight feet underground that was the reading that it 170 00:15:55,470 --> 00:16:02,380 gave due to work through the way it works and so said oh I'm below 30 feet I have to 171 00:16:02,380 --> 00:16:06,060 reduce the thrust to idle and that's what it did although it was still a couple 172 00:16:06,060 --> 00:16:10,590 hundred feet high and the pilots didn't notice early enough and let the 173 00:16:10,590 --> 00:16:18,060 speed decay and the wing stalled and crashed the airplane crashed and the nine 174 00:16:18,060 --> 00:16:23,310 people died it was moderately only a moderately hard crash so most people 175 00:16:23,310 --> 00:16:29,260 survived actually though it was still a problem and the way the auto in auto 176 00:16:29,260 --> 00:16:34,639 throttle system works in this case if the thrust levers had been static this 177 00:16:34,639 --> 00:16:39,040 wouldn't have happened because the pilots were pushed the thrust levers above a 178 00:16:39,040 --> 00:16:44,949 certain detent and it wouldn't have reduced thrust automatically again so it's 179 00:16:44,949 --> 00:16:52,260 very hard to say which system in total is better you can count the accidents maybe 180 00:16:52,260 --> 00:16:57,620 in which it played a roll but there are so few they're just really less than a 181 00:16:57,620 --> 00:17:01,610 handful in each case so they're not statistically significant so you can't 182 00:17:01,610 --> 00:17:06,780 really say by statistics alone which system is better than the other they both 183 00:17:06,780 --> 00:17:12,959 have their own problems and this is one of the decisions as engineers that you really 184 00:17:12,959 --> 00:17:19,540 can't make a decisive argument for so one manufacturer chooses one and the other 185 00:17:19,540 --> 00:17:24,780 chooses the other and there's another one is asiana flight 214 at San Francisco many 186 00:17:24,780 --> 00:17:32,020 of you may remember that. only three people were killed in this one because it 187 00:17:32,020 --> 00:17:37,510 really burned out only after the crash after everyone had evacuated and so the 188 00:17:37,510 --> 00:17:42,250 auto throttles didn't work as expected in this case the pilots thought oh the auto 189 00:17:42,250 --> 00:17:46,220 throttles will hold the speed we don't have to worry about that as far as I 190 00:17:46,220 --> 00:17:52,220 remember there were five pilots in the cockpit and when finally someone noticed 191 00:17:52,220 --> 00:17:56,550 and pushed the throttles forward it was already too late the engines take that 192 00:17:56,550 --> 00:18:00,901 time to spool up the legal requirement is that they may take up to eight seconds to 193 00:18:00,901 --> 00:18:07,890 spool up from idle to the necessary power to go around and there wasn't enough time 194 00:18:07,890 --> 00:18:11,540 for that because after the engines have spooled up the airplane also still has to 195 00:18:11,540 --> 00:18:16,800 accelerate to get back to flying speed again so in this case again the wings 196 00:18:16,800 --> 00:18:23,760 stalled the airplane crashed just short of the runway and three people died. and the 197 00:18:23,760 --> 00:18:28,090 third case was even one when nothing was wrong with the airplane except you could 198 00:18:28,090 --> 00:18:34,280 argue it was a design flaw but it was working as designed people who were going 199 00:18:34,280 --> 00:18:38,710 to fly the aircraft learned how the system worked learned everything about it 200 00:18:38,710 --> 00:18:47,210 hopefully and so more training may perhaps be the answer that is one thing system 201 00:18:47,210 --> 00:18:54,670 knowledge, two. crew resource management has been a big thing in previous decades 202 00:18:54,680 --> 00:19:00,630 that the pilot command in command is not a dictator on the airplane he has to listen 203 00:19:00,630 --> 00:19:07,070 to the others to the other pilot even though he has ultimate authority in decision. 204 00:19:11,710 --> 00:19:16,840 so do pilots always screw up if the automation fails? no luckily not if 205 00:19:16,840 --> 00:19:22,300 other systems fail in this case not the automation really but there are two 206 00:19:22,300 --> 00:19:26,090 cases which I would briefly mentioned Chesley Sullenberger everybody knows about 207 00:19:26,090 --> 00:19:33,100 him the movie has just been out the ditching in the Hudson superb pilot great 208 00:19:33,100 --> 00:19:39,100 decision making to find the biggest flat surface in the area to pull it down and 209 00:19:39,100 --> 00:19:48,410 Peter Burkhill he'll many other so who knew about Peter Burkhill? A few. he was 210 00:19:48,410 --> 00:19:53,000 the one saved about as many people as Sullenberger when on approach to london 211 00:19:53,000 --> 00:19:59,650 heathrow both engines lost thrust most of the thrust anyway and he managed to put it 212 00:19:59,650 --> 00:20:04,340 down within the airport but short of the runway it was a crash landing the airplane 213 00:20:04,340 --> 00:20:09,170 was destroyed but nobody died so it was a pretty good outcome. 214 00:20:14,406 --> 00:20:16,840 so airplanes are one thing, another thing are cars 215 00:20:19,480 --> 00:20:26,250 and anyone here has a self-driving car? Or at least a 216 00:20:26,250 --> 00:20:33,590 lane assist or something? Not many so not many people don't trust these newfangled 217 00:20:33,590 --> 00:20:41,720 systems I guess. one of the big differences is that pilots who are going 218 00:20:41,720 --> 00:20:47,400 to fly highly automated aircraft have to take a long training course beyond their 219 00:20:47,400 --> 00:20:54,200 pilot's license to learn the specifics of operating this specific aircraft and 220 00:20:54,200 --> 00:21:01,680 maintenance is very highly controlled and regulated so that's another thing. and the 221 00:21:01,680 --> 00:21:05,880 things for cars in general if something's wrong with the engine you can just pull 222 00:21:05,880 --> 00:21:13,210 over to the right and stop in most cases and cars cannot just take off and take 223 00:21:13,210 --> 00:21:18,220 evasive action in the third dimension and 224 00:21:18,220 --> 00:21:22,270 there are lots and lots of obstacles on the ground there are trees 225 00:21:22,270 --> 00:21:29,710 cars people houses everything whereas the air is mostly empty not entirely 226 00:21:30,050 --> 00:21:38,180 air-to-air collisions happen midair collisions do happen but they are very very few. in the 227 00:21:38,180 --> 00:21:45,760 automatic systems in the self-driving cars or the autonomous cars that we have today 228 00:21:45,760 --> 00:21:53,370 require constant monitoring and if the systems work too well then drivers may 229 00:21:53,370 --> 00:21:58,440 actually forget about it and think they are perfect and let their attention 230 00:21:58,440 --> 00:22:07,080 wander. pilots sometimes are prone to do that as well but the thing is that in 231 00:22:07,080 --> 00:22:13,900 Cruise in cruise flight if the automatics drop out the pilots have on the order of 232 00:22:13,900 --> 00:22:20,340 minutes to react really at least several seconds whereas on a road car if the 233 00:22:20,340 --> 00:22:26,450 automatics drop out and you're in a curve you have fractions of a second to save the 234 00:22:26,450 --> 00:22:29,390 car with the current state of the technology. 235 00:22:35,870 --> 00:22:41,160 some of you probably have heard about the trolley problem or trolley-ology as it's sometimes called. 236 00:22:44,890 --> 00:22:49,280 it basically boils down to that a fully autonomous car a highly 237 00:22:49,280 --> 00:22:56,110 automated car may eventually have to make the decision between killing the occupants 238 00:22:56,110 --> 00:23:03,740 and killing people on the road. and I think that is fundamentally an unsolvable 239 00:23:03,740 --> 00:23:11,410 ethical problem that we cannot just leave to the engineers or the car manufacturers 240 00:23:11,410 --> 00:23:15,860 to decide that maybe the occupants are always more important than people on the 241 00:23:15,860 --> 00:23:21,050 road? what if there's only one person in the car and there's a crowd on the road 242 00:23:21,050 --> 00:23:24,710 and you have to decide between steering on the car into to the tree and killing the 243 00:23:24,710 --> 00:23:33,080 sole occupant or killing several people that are in front of the car these are 244 00:23:33,080 --> 00:23:42,760 situations that may actually happen. so I really can't see what the right answer is 245 00:23:42,760 --> 00:23:49,060 to that if there is one and maybe there isn't one. some engineers have actually 246 00:23:49,060 --> 00:23:54,020 suggested that making a random decision in that case is the answer. I'm not too sure 247 00:23:54,020 --> 00:24:02,150 about that either but whatever the decision the software takes at that moment 248 00:24:03,080 --> 00:24:10,630 then people will die and they will take the blame either way and we don't know yet 249 00:24:10,630 --> 00:24:13,120 how that's going to turn out in front of the courts. 250 00:24:17,990 --> 00:24:20,200 so automation is hard to get 251 00:24:20,200 --> 00:24:24,760 right and in some cases self-driving cars it may be impossible to get it absolutely 252 00:24:24,760 --> 00:24:33,250 right. which stage is the safest for the systems to be in and at what time who 253 00:24:33,250 --> 00:24:41,350 knows it's very very hard to get it right even in limited systems such as airplanes 254 00:24:41,350 --> 00:24:47,340 and what to display to the operators and when in many cases it would help the 255 00:24:47,340 --> 00:24:55,030 pilots a lot when the automation drops out to know intimate details of how the system 256 00:24:55,030 --> 00:25:01,000 works internally. airbus has some logic diagrams in their pilots handbook but they 257 00:25:01,000 --> 00:25:06,210 are labeled 'for info' which means they are not required for any exams it's just 258 00:25:06,210 --> 00:25:11,790 interesting to know but in case of the logic for extension of the ground spoilers 259 00:25:11,790 --> 00:25:15,840 it's quite helpful to know which conditions exactly have to be satisfied 260 00:25:15,840 --> 00:25:17,380 for the ground spoilers to deploy. 261 00:25:21,400 --> 00:25:24,400 but some of these problems I think cannot 262 00:25:24,400 --> 00:25:31,360 be left to engineers and scientists alone and we need psychologists and maybe 263 00:25:31,360 --> 00:25:38,040 sociologists other people who know about the psyche of people who know about how 264 00:25:38,040 --> 00:25:45,100 people think how people react how people process information to make good 265 00:25:45,100 --> 00:25:51,850 engineering design decisions to build safer systems. and as I said some of the 266 00:25:51,850 --> 00:25:59,910 fundamental ethical problems may turn out to remain unsolvable. 267 00:25:59,910 --> 00:26:02,980 thank you I think we have a little bit of time for questions 268 00:26:02,980 --> 00:26:12,340 [Applause] 269 00:26:12,350 --> 00:26:14,700 MC: yes we actually do we have some time 270 00:26:14,700 --> 00:26:19,200 for questions and we're gonna start with the internet if there are any questions no 271 00:26:19,200 --> 00:26:21,450 there are not then it's microphone number three 272 00:26:21,450 --> 00:26:27,470 Q: yes you mentioned the ethical problem of the decision making the trolley problem 273 00:26:27,470 --> 00:26:33,210 so whenever this comes up regarding automated driving systems whether it be 274 00:26:33,210 --> 00:26:39,680 flight control or car driving I always get a little bit mad when philosophers come up 275 00:26:39,680 --> 00:26:44,840 with that the there is one decisive decision you can make and that is the 276 00:26:44,840 --> 00:26:50,680 whole thing should act predictably especially in road traffic the uttermost 277 00:26:50,680 --> 00:26:58,160 importance is that all participants behave predictably swerving out of lane is the 278 00:26:58,160 --> 00:27:01,730 most dangerous thing you can do H: And what's your question? 279 00:27:01,730 --> 00:27:08,210 Q: and if you have to make this decision people say you 280 00:27:08,210 --> 00:27:13,000 have to make a decision then I say no there is a definitive safe state that is 281 00:27:13,000 --> 00:27:20,800 drive with enough distance to the guy in front of you don't tailgate don't speed up 282 00:27:20,800 --> 00:27:23,920 because if you're a regular driver MC: no no please ask your question 283 00:27:23,920 --> 00:27:30,600 Q: okay question is why are people always saying it's ethically not decisive 284 00:27:30,600 --> 00:27:34,790 decidable? BS: it isn't because if if just keeping 285 00:27:34,790 --> 00:27:39,220 enough distance we solve all problems that would be that would be fine but cars are 286 00:27:39,220 --> 00:27:44,950 not the only participants in traffic there are people right and they can just jump in 287 00:27:44,950 --> 00:27:51,480 front of a car. that is not predictable, yeah you can you can require people to 288 00:27:51,480 --> 00:27:54,900 behave predictable but good luck with that. 289 00:27:54,900 --> 00:28:01,712 Q: I would like to counter that [Applause] 290 00:28:01,712 --> 00:28:07,240 MC: okay I'm sorry there's not much room for discussion right now but microphone 291 00:28:07,240 --> 00:28:15,260 number two please ask a concise question Q: okay let me try so you said about 292 00:28:15,260 --> 00:28:19,740 automation in airplanes that whenever there is a small malfunction the autopilot 293 00:28:19,740 --> 00:28:24,820 will disconnect and expect the pilots to fix the situation right so it is my- 294 00:28:24,820 --> 00:28:27,970 BS: yeah it's not not the smallest problem but some yeah 295 00:28:27,970 --> 00:28:32,290 Q: okay so but it is my understanding that the pilots are still expected to follow 296 00:28:32,290 --> 00:28:39,300 procedures and not make any random gut decisions in most cases. my question is do 297 00:28:39,300 --> 00:28:44,620 you have statistics when the standard procedures were actually not applicable in 298 00:28:44,620 --> 00:28:48,650 how many cases and in how many of these cases did the pilots actually managed to 299 00:28:48,650 --> 00:28:53,970 save the flight? BS: no I'm not aware of any statistics and 300 00:28:53,970 --> 00:29:00,210 one of the problems with that is that in general the data recorder is only read 301 00:29:00,210 --> 00:29:04,260 when there was an accident and it is strictly off-limits in all other 302 00:29:04,260 --> 00:29:10,070 circumstances some airplanes have a quick access data recorder which they can 303 00:29:10,070 --> 00:29:16,300 routinely read but only anonymized so and I don't think the airplanes published 304 00:29:16,300 --> 00:29:21,420 statistics about that MC: okay last question microphone number 305 00:29:21,420 --> 00:29:25,230 four please Q: yeah I'm just I just want to bring this 306 00:29:25,230 --> 00:29:32,150 back to this sort of the IT security part where what I find very good about about 307 00:29:32,150 --> 00:29:37,970 the way accidents are handled in in aviation is that the report is completely 308 00:29:37,970 --> 00:29:41,480 public so if you want to read you know the Challenger cut this show if you can you 309 00:29:41,480 --> 00:29:44,650 can actually read all the technical details and all that all the stuff that 310 00:29:44,650 --> 00:29:53,660 happened and and all that information is there and the question is is why is this 311 00:29:53,660 --> 00:29:58,020 not happening in the IT sector where clearly millions of people are being 312 00:29:58,020 --> 00:30:05,810 affected and somehow you haven't reached this stage where the the the data and the 313 00:30:05,810 --> 00:30:10,530 analysis of the data is public so we can all learn from it and get better as it has 314 00:30:10,530 --> 00:30:14,630 been you know the way BS: I think the short answer is excuse me 315 00:30:14,630 --> 00:30:17,350 Q: no it's good BS: I think the short answer is because 316 00:30:17,350 --> 00:30:24,000 there is no legal requirement and if there weren't for accident reports to be 317 00:30:24,000 --> 00:30:26,820 distributed then many airlines wouldn't do it . 318 00:30:26,820 --> 00:30:31,410 Q: but why? It's very clear [???] BS: it's because it's embarrassing if you 319 00:30:31,410 --> 00:30:34,380 have an accident it's basically the thing I think 320 00:30:34,380 --> 00:30:36,840 MC: ok very last question microphone number one please 321 00:30:36,840 --> 00:30:42,970 Q: hey so one of the reasons we have automation in aircraft in the first place 322 00:30:42,970 --> 00:30:47,270 is to reduce pilot workload where too high pilot workload is a major cause of 323 00:30:47,270 --> 00:30:53,200 accidents it seems like one of the issues we're talking about here is that in a 324 00:30:53,200 --> 00:30:57,130 situation where something's gone wrong the presence of that automation are needed to 325 00:30:57,130 --> 00:31:01,000 understand it means you've got a higher pilot workload in that situation the 326 00:31:01,000 --> 00:31:06,570 question what is it doing now what's the industry sort of approach to that effect 327 00:31:06,570 --> 00:31:13,280 and what do you think about that? BS: I think the traditional approach is to 328 00:31:13,280 --> 00:31:18,230 just pile on more automation so then if that fails the pilot has an even higher 329 00:31:18,230 --> 00:31:25,990 workload but the current thing is that manufacturers and the airlines go back 330 00:31:25,990 --> 00:31:33,270 very very slowly to letting the pilot hand fly more often and for a long time the the 331 00:31:33,270 --> 00:31:38,750 mantra was use automation whenever possible the highest level of automation 332 00:31:38,750 --> 00:31:43,420 that is appropriate for the situation so only the takeoff and touchdown were flown 333 00:31:43,420 --> 00:31:50,630 by hand and now it is very often use the appropriate level of automation and that 334 00:31:50,630 --> 00:31:55,870 means if there's not very high workload and not a lot of traffic then hand fly the 335 00:31:55,870 --> 00:32:06,410 approach for example. so to to keep in good practice and right to maintain 336 00:32:06,410 --> 00:32:10,064 proficiency for all situations hopefully. 337 00:32:13,679 --> 00:32:17,290 MC: thank you and please give a warm hand of applause for Bernd Sieker. 338 00:32:17,300 --> 00:32:21,984 [Applause] 339 00:32:21,984 --> 00:32:26,664 [Music] 340 00:32:26,664 --> 00:32:46,000 subtitles created by c3subtitles.de in the year 2018. Join, and help us!