European law protects you from automated decisions. Which sounds like a minor thing, but if an
automated system is making a meaningful decision about your life -- whether you get a pay rise,
or if you get a formal warning for clocking in late at work -- you must be told that it
was an automated decision, and you must be able to get it reviewed by a human. At least,
in Europe. And importantly, you can tell a company to never make automated decisions
about you and they must comply with that. Now that doesn't apply to little things like
online personality quizzes, unless that quiz decides -- without a human in the loop -- whether
you're a safe person to lend money to.
But those are all fairly benign automated systems, even if it might not feel like it
when you've just been turned down for a loan.
What about automated systems that are meant to be harmful to humans?
I've filmed at this tank once before. It's an old Soviet T-34 battle tank, put here in
London two decades ago by an angry local resident who was denied permission to build on this
bit of land... but did have permission to store vehicles here. Or maybe he got permission
to put a "tank" there, because the local council thought he was talking about some sort of
storage tank, not an actual T-34. The story's got a bit muddled over the last twenty years or so.
But this tank is designed for people. It's designed for human soldiers making human decisions.
And that concept might not be the case for very much longer.
See, the US Army just put out a report called Visualising the Tactical Ground Battlefield
in the Year 2050. Which sounds like a really dull comic book, but it's actually a pretty
good read, and they have some things to say about automated decision making.
The US Army believes that humans will be incapable of handling all the information that'll be
thrown at them in the height of a future battle. Computer processing power, they say, won't
be a limiting factor, and as humans grow more accustomed to automated decision making, of
course we'll be happy to use that in the battlefield too. If you have a soldier who's happy with
flipping the autopilot on their car, of course they're going to be happy doing exactly the
same for their tank and for their gun.
"Battle rhythm," -- and I'm quoting the report directly here,
"Battle rhythm will increase to the point that, in many instances, humans will no longer
be able to be 'in the loop,' but will instead need to operate 'on the loop', where 'humans
can only observe the behaviors that are taking place'... and 'act after the fact or in anticipation
of expected behaviors.'"
So just to be clear: the US Army believes, in the next few decades, that they will have
autonomous weapons which will have patterns of behaviour that an engineer is going to
have to try and debug while it is firing live ammunition. If you're a programmer, working
on open-source computer vision, or collision avoidance, or, say, something like flocking
behaviour algorithms, there is a chance that your code will end up as part of that. And
if you've got moral objections: well, good for you, but it's probably not going to stop anyone.
And I suspect all the legislation that Europe can muster won't do a thing when you've got
one of these, quite literally, calling the shots.
[Translating these subtitles? Add your name here!]