CSCI3390 Home Page



CSCI3390-Topics in Computer Science

Computability and Complexity

Fall, 2018



General Course Information What this Course is About Required Background Textbook
Required Work Grading The laptop-free classroom
Detailed Course Schedule

General Course Information


Instructor:

Howard Straubing
Computer Science Department
251 St. Mary's South
tel.:617-552-3977
e-mail: straubin@cs.bc.edu
Office Hours: MW 2-4

The class meets Tuesdays and Thursdays in 130 Merkert   at 1:30.



What this Course is About

What can be computed?  What can a computer--the one on your desk or on your lap or in your pocket, or computers that have yet to be built-- do?  What can't a computer do? Are there really problems that cannot be solved by any algorithm (and how would one ever prove such a thing)?  Are there problems that, while solvable in principle, can never be solved in practice, because a practical solution would require too much time or memory?  Is this a purely logical question, or is it a technological question?  Would computers that operate on different physical principles be able to do things that our present computers cannot?

Since the 1930's, beginning with the work of Church and Turing, mathematicians and computer scientists have been exploring these questions by creating mathematical models of computation and studying their formal properties.  Their discoveries have fostered a deeper understanding of the limitations of both computation and mathematical reasoning, aided in the development of secure (maybe) cryptographic systems that are used by all of us every day, and the prospect, at least in principle, of building quantum-mechanical computers that solve problems out of the reach of conventional computers (and also break those secure cryptographic systems).

The following is an approximate list of topics:

I don't expect that we will do everything on this list, at least not in detail.  This is a 'terminal' course, in the sense that it is not a prerequisite for anything else you will take as a Boston College undergraduate, and some of the ideas are difficult. So I am in no rush to 'cover the material':  I will slow down as necessary to make sure that as many students as possible are up to speed, and prefer to drop a topic rather than speed through something important that you might need more time with.

This is a theory course, which means that there will be a fair amount of math written on the board and in your homework assignments.  But I intend throughout to emphasize connections to practice where I can.  Most of optional project topics (see below) will incorporate elements of both mathematical theory and creating a useful program.

Required Background

On the mathematical side, either Introduction to Abstract Mathematics in the Math department, or Logic and Computation in the Computer Science department, or something equivalent, is essential. This course requires a working familiarity with abstract mathematical notation for sets and functions, and the ability to read, follow, and in some cases write mathematical proofs.  I know, of course, that a lot of this material is easily forgotten, so I will often pause to remind you what something means, but at least in terms of the basics, you should not be seeing this stuff for the first time.

You also need to have some programming background, at least the equivalent of a one-semester introductory course (for example, CS1101).  It should not matter much what programming language you studied.


Textbooks?

 There is no official course textbook.  Although much of the material is by now pretty standard, there is no one book that treats all these topics in the way I would like to.  I will post lecture notes, usually just after the class in which the relevant material is presented, so that the notes can take account of points the students may have found difficult during the lecture.

Still, it's nice to be able to read something, so here is a selection of recommended items:

Introduction to the Theory of Computation, by Michael Sipser.  If there were a textbook for the course, this would be it.  It covers the majority of the topics on the list above (and much, much else), and at what I consider an appropriate level.  The main drawback is the price:  The current (2012) 3rd edition will set you back $200 new.  If you are interested in buying something, I would advise you to look for a copy of the 2005 2nd edition.  There are used copies available on Amazon for $75 or less.  The material in Chapters 3-5, 7, and parts of Chapters 8 and 10 are all relevant.

Computational Complexity, A Modern Approach, by Sanjeev Arora and Boaz Barak.  In its totality, this book is far more advanced than what is appropriate for this course, and is more of a graduate-level textbook.  But the early chapters are about right for us, and an earlier draft is available free at Arora's website.  (The print edition is also quite reasonably priced, but only a relatively small fraction of it will be discussed in this course.)  Chapters 1, 2, parts of 4, 7 and 10, and Chapter 20 of the web draft are relevant to us (corresponding to Chapters 0,1,2,4,7,9 and 10 of the print edition.)

Computers and Intractability: A Guide to the Theory of NP-Completeness, by Michael R. Garey and David S. Johnson.  This 1979 classic, has the honor of being the most-cited reference in the computer science literature. It concentrates on just one piece (but a big piece) of our course.  It is still relevant, and not too expensive on Amazon.

The Nature of Computation, by Christopher Moore and Stephan Mertens. Big (> 900 pages), relatively new (2011), quirky (it does computational complexity before computability, unlike almost every other book on the subject), engaging and fascinating, not at all like a standard textbook.

Logic and Computation.  The free online textbook that I wrote for CS2243 will be available on the Canvas site for this course.  It is there mostly as a reference for the mathematical prerequisites, although the book ends where we start, with Turing machines and computability.

If you do use other material, you should be aware that the exact definitions of many terms (especially of Turing machines) are not completely standard--the fact that they do not have to be completely standard is an important principle in the subject.



Required Work

Homework

There will be a homework assignment approximately every week.  Most homework problems are exercises designed to build and test your understanding of the material by working through specific examples, although you will often be asked to prove something, and occasionally to code something.

All the assignments will be posted on this website, and submitted through the Canvas site for this course.  The assignment must be submitted by 11:59PM on the due date.

When preparing an assignment, you should place all the files in a folder named

First Name_Last Name_Assignment_n

(For instance, if I were a student in the course, I would name the folder for the first assignment Howard_Straubing_Assignment_1.) Compress the folder to produce a .zip file, and then upload this through the submission portal on Canvas.  Do not submit assignments as attachments to e-mail messages! 

It is easier to deal with electronic submissions than with great masses of paper.  But courses like this have homework with lots of mathematical symbol and diagrams, and are typically prepared with pencil and paper. So let me suggest several alternatives.

Midterm exams


There will be two one-hour tests, one late in February and the other late in March.

Project

In lieu of a final exam, you will be asked to complete a project or a paper.   A project may seem  like a weird idea for a theory course, but I have a number of suggestions that integrate both understanding of the underlying theory and production of a working piece of software.  There is also the option of doing an entirely theoretical project, in which you write a report that presents detailed proofs and examples.  Projects may be completed individually or in pairs; keep in mind that two people are expected to do twice the work of one!

(A link to suggested project topics will appear here early in the semester.)


Academic Integrity

While I encourage you to discuss homework problems both among yourselves and with me, when it comes down to finally sitting down and preparing the completed assignment, you are required to work alone.  Thus it is acceptable to learn from another student the general idea for writing program code to perform a particular task, or the technique for solving a mathematical problem, but unacceptable for two students to prepare their assignments together and submit what are essentially two copies of identical work.  If you have any uncertainty about the application of this policy, please check with me.

It is also acceptable to consult other sources than the course lecture notes or the recommended reading. You may very well find the idea for solving a problem (or, in some cases, even the solution itself) in something you read.  If that is the case, please let me know what you have found and where you found it.  Generally speaking, it is okay to incorporate an idea that you discovered elsewhere into your work (but check with me first, and cite your source), but it is plagiarism to copy a significant portion of what you find and present it as your own work.

Failure to comply with these guidelines will be considered a violation of the University policies on academic integrity. Please make sure that you are familiar with these policies.

Services for Students with Disabilities

If you are a student with a documented disability seeking reasonable accommodations in this course, please contact Kathy Duggan, (617)552-8093, dugganka@bc.edu, at the Conors Family Learning Center, regarding learning disabilities and ADHD.  For all other types of disabilities, including temporary disabilities, contact Rory Stein, (617) 552-3470, steinr@bc.edu, in the Disability Services Center.  Advance notice and appropriate documentation are required for accommodations.


 

Eyes Front!

My colleague Bob Muller calls this policy the 'laptop-free classroom'.

A successful class requires your attention, engagement, and participation.  You need to be prepared to ask and answer questions during the lectures, and to attend to the questions and answers of your fellow students and of the instructor.  That screen open to your e-mail or Facebook page distracts not only you, but the students sitting behind you. For this reason, open laptops are not permitted in the classroom.  The same applies to smaller screens:  put the phone away, you can read and write your texts after class.

You say you're taking notes on that laptop instead of looking at reddit? You should be aware that (a) it is very difficult to take notes for this class on a computer, given the large number of mathematical symbols. formulas and diagrams that we use; and (b) taking notes by hand is better for you

It is acceptable to take notes on a tablet computer with a stylus.


Grading

To be eligible for  a passing grade in this course, you must complete at least 60% of the assignments in a satisfactory manner. Your grade will then be computed from your performance on assignments, quizzes and midterm, and final exam.  The assignments will account for about 35% of the grade, each of the two tests for about 20%, and the project for about 25%.  (These ratios are subject to minor adjustments, hence all the "abouts".)


Detailed Course Schedule

This grid will grow and change as the semester progresses. 
Note that there is no class October 9 or November 22.  November 20 is a regular class day.
Date
Topic
Lecture notes and other readings

Homework
August 28-30
Introduction.

Encoding problems as strings.
The Turing machine model.lecture6.pdf
Lecture 0: Overview

Lecture 1: What is a computation? Turing machines

How to use the Turing machine simulator.

reverse.tm  - a typical file for the simulator.

tm.py - the simulator program (Python 3 version)
tmpy2.py - the simulator program (Python 2 version)


Turing's 1936 paper, 'On computable numbers, with an application to the Entscheidungsproblem', introducing Turing machines.  Of most interest for us is the very readable Section 9, where Turing attempts to justify why the machine model captures the intuitive notion of what a 'computer'  (by which he meant a person!) can do. 


Resources for learning LaTeX :

The not-so-short introduction to LaTeX

Wikibook on LaTeX


Assignment 1, due Thursday, September 6.

LaTeX source for assignment.

Graphics files from the assignment (you need these to get the LaTeX source to compile):

graphencoding.jpg
hw1tm.jpg

Solutions
September 4-11
The Turing machine model, continued.
The Church-Turing thesis.


Lecture 2: Turing machines for decision problems.

equalasbs.tm - specification file for the machine that tests for equal numbers of a's and b's

Lecture 3: Church-Turing thesis; equivalence of multi-tape and  one-tape Turing machines.
Assignment 2, due Tuesday, September 18.

LaTeX source for assignment

Graphics files for the images in the assignment:

Figure 1
Figure 2
Figure 3
Figure 4

Assignment 2 Solutions

The Turing machine specifications for the assignment:

Equal halves (simpler version)
Move over one cell
Insert blank cells throughout input
Reversal with a unidirectional tape
September 13-21

Universal Turing machine
Undecidability of the Halting Problem.
Reductions.
A sampler of undecidable problems about Turing machines (and a few decidable ones).
Lecture 4: Closure properties, additional computational problems

Lecture 5: Universal Turing Machine

Bedtime paradox

Lecture 6: An undecidable problem.

Lecture 7:  Undecidable problems about Turing machines.
Assignment 3, due Tuesday, October 2.

LaTeX source for the assignment

Sudoku figure

Rush Hour figure
September 18-20
Reducibility.

A sampler of undecidable problems that are not about Turing machines and computer programs.
Lecture 8:  A sampler of undecidable problems



September 25-27
Undecidability of the Tiling Problem (special case)

Undecidability of arithmetic; Goedel's Theorem


Lecture 9: Proof of undecidability for string rewriting systems and the constrained tiling problem.

Can't Decide? Undecide!  An very readable article from the Notices of the American Mathematical Society on undecidable problems.

Lecture 10: Goedel's Theorem


October 2-4
More on reductions.
The first midterm from a previous version of the course

...with solutions

October 11
First test.


October 16-18
Alternate models of computation (recursive functions, lambda calculus)


October 23-25
The class P.

Nondeterministic Turing machines and the class NP.


October 30-November 1
Propositional Logic and Satisfiability


November 6-November 8
NP-completeness and the Cook-Levin Theorem. The P=NP problem

An assortment of NP-complete problems


November 13-15
Practical approaches to NP-complete problems: Satisfiability solvers.


November 20
Probabilistic algorithms in number theory; primality testing and public-key cryptography.


November 27
Quantum computing
Second test, Nov. 29


December 4-6
More quantum computing. Shor's Theorem