Impossible Programs

Impossible Programs

Every aspect of our lives has been transformed by the invention of general-purpose programmable computers. As a result, it’s tempting to believe that computers can solve any logical or mathematical problem; that if we throw enough time, money and nerds at a question, we can produce a program which answers it.

Unfortunately the universe is never that convenient. There are hard theoretical limits on what programs are capable of doing, and there will always be easily-stated problems which are impossible for any computer to solve.

This talk uses Ruby to tell a nail-biting, maths-free story about the source of a computer’s power, the inevitable drawbacks of that power, and the impossible programs which lie at the heart of uncomputability.

Given at Scottish Ruby Conference 2013 (http://lanyrd.com/2013/scotruby/) and GoGaRuCo 2013 (http://lanyrd.com/2013/gogaruco/). There’s a video of this talk at http://codon.com/impossible-programs. This talk is adapted from chapter 8 of Understanding Computation (http://computationbook.com/).

Cd9b247e4507fed75312e9a42070125d?s=128

Tom Stuart

May 12, 2013
Tweet

Transcript

  1. IMPOSSIBLE PROGRAMS @tomstuart / Scottish Ruby Conference / 2013-05-12

  2. WE ARE COMPUTER PROGRAMMERS

  3. WRITE TONS OF COMPUTER PROGRAMS WE

  4. IT’S OUR JOB

  5. PROGRAMS ARE PRETTY GOOD I SUPPOSE

  6. CAN’T DO PROGRAMS EVERYTHING

  7. WHY NOT?

  8. VAGUE DIFFICULT EXPENSIVE IMPOSSIBLE

  9. ho can a PROGRAM be IMPOSSIBLE?

  10. WE DEMAND UNIVERSAL SYSTEMS

  11. We can translate any Python program into Ruby. We can

    translate any Ruby program into Python. We can implement a Python interpreter in Ruby. We can implement a Ruby interpreter in Python. We can implement a JavaScript interpreter in Ruby. We can implement a Ruby interpreter in JavaScript. We can implement a Turing machine simulator in Ruby. We can implement a Ruby interpreter as a Turing machine.
  12. JavaScript Ruby Python Lambda calculus Turing machines SKI calculus Tag

    systems Partial recursive functions Game of Life Rule 110 C++ Haskell Lisp Register machines Magic: The Gathering C Java XSLT
  13. Universal systems can run software. We don’t just want machines,

    we want general-purpose machines.
  14. PROGRAMS ARE DATA

  15. >> puts 'hello world' hello world => nil >> program

    = "puts 'hello world'" => "puts 'hello world'" >> bytes_in_binary = program.bytes.map { |byte| byte.to_s(2).rjust(8, '0') } => ["01110000", "01110101", "01110100", "01110011", "00100000", "00100111", "01101000", "01100101", "01101100", "01101100", "01101111", "00100000", "01110111", "01101111", "01110010", "01101100", "01100100", "00100111"] >> number = bytes_in_binary.join.to_i(2) => 9796543849500706521102980495717740021834791
  16. >> number = 9796543849500706521102980495717740021834791 => 9796543849500706521102980495717740021834791 >> bytes_in_binary = number.to_s(2).scan(/.+?(?=.{8}*\z)/)

    => [ "1110000", "01110101", "01110100", "01110011", "00100000", "00100111", "01101000", "01100101", "01101100", "01101100", "01101111", "00100000", "01110111", "01101111", "01110010", "01101100", "01100100", "00100111"] >> program = bytes_in_binary.map { |string| string.to_i(2).chr }.join => "puts 'hello world'" >> eval program hello world => nil
  17. UNIVERSAL SYSTEMS + PROGRAMS ARE DATA = INFINITE LOOPS

  18. Every universal system can simulate every other universal system, including

    itself. More specifically: every universal programming language can implement its own interpreter.
  19. def evaluate(program, input) # parse program # evaluate program on

    input while capturing output # return output end
  20. >> evaluate('print $stdin.read.reverse', 'hello world') => "dlrow olleh"

  21. def evaluate(program, input) # parse program # evaluate program on

    input while capturing output # return output end def evaluate_on_itself(program) evaluate(program, program) end
  22. >> evaluate_on_itself('print $stdin.read.reverse') => "esrever.daer.nidts$ tnirp"

  23. def evaluate(program, input) # parse program # evaluate program on

    input while capturing output # return output end def evaluate_on_itself(program) evaluate(program, program) end program = $stdin.read if evaluate_on_itself(program) == 'no' print 'yes' else print 'no' end does_it_say_no.rb
  24. $ echo 'print $stdin.read.reverse' | ruby does_it_say_no.rb no $ echo

    'print "no" if $stdin.read.include?("no")' | ruby does_it_say_no.rb yes $ ruby does_it_say_no.rb < does_it_say_no.rb ???
  25. does_it_say_no.rb yes no never finish other output? does_it_say_no.rb ✘ ✔

    ✘ ✘
  26. Ruby is universal so we can write #evaluate in it

    so we can construct a special program that loops forever
  27. s here' one PROGRAM IMPOSSIBLE

  28. Sometimes infinite loops are bad. We could remove features from

    Ruby until there’s no way to cause an infinite loop.
  29. remove while / until / loop, only allow iteration over

    finite data structures to prevent -> x { x[x] }[-> x { x[x] }] only allow a method to call other methods whose names come later in the alphabet • No unlimited iteration • No procs • No recursive method calls • No blocking I/O • ...
  30. Call this language “Rub”. It must be impossible to write

    a Rub interpreter in Rub.
  31. if we could write #evaluate in Rub so it must

    be impossible to write #evaluate in Rub then we could use it to construct a special program that loops forever but Rub doesn’t let you write programs that loop forever
  32. (That’s weird, because a Rub interpreter always finishes eventually, so

    it feels like we should be able to write it in Rub.)
  33. We could write a Rub interpreter in some other more

    powerful language that also forbids infinite loops... ...but that language can’t implement its own interpreter either.
  34. ABOUT RUBY? WHAT okay but

  35. #evaluate is an impossible program for Rub, which means that

    Rub isn’t universal. Universal systems like Ruby have impossible programs too.
  36. input = $stdin.read puts input.upcase This Ruby program always finishes.*

    * assuming STDIN is finite & nonblocking
  37. input = $stdin.read while true # do nothing end puts

    input.upcase This Ruby program always loops forever.
  38. Can we write a Ruby program that can decide this

    in general?
  39. input = $stdin.read output = '' n = input.length until

    n.zero? output = output + '*' n = n - 1 end puts output
  40. input = $stdin.read output = '' n = input.length until

    n.zero? output = output + '*' n = n - 2 end puts output
  41. require 'prime' def primes_less_than(n) Prime.each(n - 1).entries end def sum_of_two_primes?(n)

    primes = primes_less_than(n) primes.any? { |a| primes.any? { |b| a + b == n } } end n = 4 while sum_of_two_primes?(n) n = n + 2 end print n
  42. def halts?(program, input) # parse program # analyze program #

    return true if program halts on input, false if not end
  43. >> halts?('print $stdin.read', 'hello world') => true >> halts?('while true

    do end', 'hello world') => false
  44. def halts?(program, input) # parse program # analyze program #

    return true if program halts on input, false if not end def halts_on_itself?(program) halts?(program, program) end program = $stdin.read if halts_on_itself?(program) while true # do nothing end end do_the_opposite.rb
  45. $ ruby do_the_opposite.rb < do_the_opposite.rb

  46. do_the_opposite.rb eventually finish loop forever do_the_opposite.rb ✘ ✘

  47. Every real Ruby program must either loop forever or not,

    but whichever happens, #halts? will be wrong about it. do_the_opposite.rb forces #halts? to give the wrong answer.
  48. if we could write #halts? in Ruby so it must

    be impossible to write #halts? in Ruby then we could use it to construct a special program that forces #halts? to give the wrong answer but a correct implementation of #halts? would always give the right answer
  49. This is the halting problem.

  50. WHO CARES? okay but

  51. We never actually want to ask a computer whether a

    program will loop forever. But we often want to ask computers other questions about programs.
  52. def prints_hello_world?(program, input) # parse program # analyze program #

    return true if program prints "hello world", false if not end
  53. >> prints_hello_world?('print $stdin.read.reverse', 'dlrow olleh') => true >> prints_hello_world?('print $stdin.read.upcase',

    'dlrow olleh') => false
  54. def prints_hello_world?(program, input) # parse program # analyze program #

    return true if program prints "hello world", false if not end def halts?(program, input) hello_world_program = %Q{ program = #{program.inspect} input = $stdin.read evaluate(program, input) # evaluate program, ignoring its output print 'hello world' } prints_hello_world?(hello_world_program, input) end
  55. if we could write #prints_hello_world? in Ruby so it must

    be impossible to write #prints_hello_world? in Ruby then we could use it to construct a correct implementation of #halts? but it’s impossible to correctly implement #halts? in Ruby
  56. Not only can we not ask “does this program halt?”,

    we also can’t ask “does this program do what I want it to do?”.
  57. This is Rice’s theorem: Any interesting property of program behaviour

    is undecidable.
  58. WHY HAPPEN? DOES THIS

  59. We can’t look into the future and predict what a

    program will do. The only way to find out for sure is to run it. But when we run a program, we don’t know how long we have to wait for it to finish. (Some programs never will.)
  60. Any system with enough power to be self-referential can’t correctly

    answer every question about itself. We need to step outside the self-referential system and use a different, more powerful system to answer questions about it. But there is no more powerful system to upgrade to.
  61. HOW COPE? CAN WE

  62. • Ask undecidable questions, but give up if an answer

    can’t be found in a reasonable time. • Ask several small questions whose answers provide evidence for the answer to a larger question. • Ask decidable questions by being conservative. • Approximate a program by converting it into something simpler, then ask questions about the approximation.
  63. Stuart Understanding Computation From Simple Machines to Impossible Programs Tom

    Stuart Understanding Computation THE END SCOTCOMP (50% off ebook, 40% off print http://computationbook.com/ @tomstuart / tom@codon.com