• Makespan: 3/2, 4/3
• Center selection: 2
• Vertex cover: 2

The goal is to achieve a (1 + e) approximation.

Knapsack Problem:

We can carry at most W weight. We have n items to choose from, each with weight w_i and a value v_i. Our goal is to maximize value without exceeding weight boundary W. Using Dynamic Programming, we can define:

Opt(i, V) = smallest weight possible using items {1, …, i} and choosing value >= V.

Given the Opt(*, *) function and W, we can search the opt-table to find the largest V for which opt(n, V) <= W:

V = sum(v_i in S) <= sum(v_i for all i) <= nv^*, where v^* = max v_i

Therefore, the algorithm runs in time O(n²v^*).

Algorithm notes:

If the nth item is not in opt solution, then Opt(n, V) = Opt(n-1, V). If the nth item is in the optimal solution, then Opt(n, V) = w_n + Opt(n-1, V-V_n)–so, just take the minimum:

Opt(n, V) = min({Opt(n-1, V), w_n + Opt(n-1, V-V_n})

There are some table bounds to consider:

Opt(1, V) = w₁ if v₁ >= V, else infinity
Opt(k, 0) = 0
Opt(k, < 0) = 0

Finally, we can write the algorithm outline:

Knapsack0Approx(e):
    b = (e/n) * max(v_i)
    for all i v_i^{^} = ceil(v_i/b)
    solve problem with v_i^{^} via DP outline above
    return result set S

Linear Programming:

A set of linear inequalities define a feasible region, for example:

x + y >=0
x – y >= 0
x <= 5
y <= 2

Or, in matrix form:

[[1,1][1,-1][1,0][0,1]] * [[x][y]] >= [[0][0][-5][-2]]

The goal is to remain the feasible region while optimizing something. In general, given {Ax = b}, a set of linear inequalities, we want to minimize c*x, i.e. find the minimum {c*x | Ax >= b}. Geometrically, this is finding the extreme point of a polytope. We can write this as a decision problem, too. Given A, b, c, and a bound B, is there a solution x such that Ax >= b and c’*x <= B?

Claim: “LP is in NP”

Given a set of values, we can check the solution in polynomial time. Rational numbers, however, require additional book keeping. Beware! Also, LP is in Co-NP and in P, using an interior point method.

VC as a LP problem:

x_i >= 0
-x_i >= -1
x_i + x_j 7gt;= 1 for all (v_i, v_j)
minimize sum(x_i)

Then, run LP but only allow integer solutions. This is called “Integer Programming” (IP). Claim: “VC <=_p IP.” Rewrite VC as an IP problem, then the solution to one implies the other. Claim: “IP is in NP.” Obvious. Therefore, IP is NP-Complete.

Approximation:

Simply round the LP solution to get integers. S = {i | i >= 1/2}, our approximate vertex cover. Then, |S| = sum(i in S) <= sum(2*x_i in S) = s * sum(x_i in S) <= 2 * sum(x_i from i = 0 to n) <= 2 * |S*|. So, we are within a factor of two of the optimal solution.

dist(*,*) must satisfy:

• dist(s, s) = 0 (reflexive)
• dist(s, t) = dist(t, s) (symmetry)
• dist(s, u) <= dist(s, t) + dist(t, u) (triangle inequality)

CS_decision <=_p CS_optimization: Optimization is NP-hard!

Suppose that we know r and want to find a solution, i.e. choosing the centers. If we already know one of the centers, uf we were to use radius 2r then any point within distance r can be used as the center, and all points will still be covered. So, pick any point s in the disk; the circle with center s and radius 2r covers all the same points.

If there is a technique that produce an answer where radius <= (2 – epsilon)r*, then p = NP. In the plane, using Euclidean distance, if radius <= (1.89 …)r*, then P = NP.

Greedy Algorithm: Let S be the set of sites and C be the set of centers. We are given r.

C = {}
while S != {}
    choose s in S
    C = C + {s}
    delete all s’ in S within 2r of s
end
if |C| <= k return C
else report “No solution for radius r possible”

Claim: “If there is a solution using k radius-r disks, then the algorithm returns a solution using <= k disks of radius 2r.

Each s chosen by the algorithm is in some disk (c*, r) for some c*. Then the circle (s, 2r) covers all the same points covered by (c*, r) plus some more. Thus, later choices of s are not covered by the circle (c*, r). Each s chosen is covered by a different c* circle. There are at most |c*| = k choices for s. The algorithm produces <= k disks of radius 2r covering all points.

Method: Use a binary search to find the right radius. First, box the set of points and find the maximum seperation between any 2 points. This is R and r <= R. We can actually do better! During the algorithm we need a point at distance >= 2r from chosen site s. Always choose the fathest point. For later steps, choose s_i farthest from existing centers.

New algorithm:

Select any site s in S, let C = {s}
while |c| < k
    select s in S that maximizes that distance from s to any point in C
    C = C + {s}
end
return C

Claim: “If there’s a solution (C*_r, r) where |C*_r|=k, then the algorithm returns a solution (C, 2r) where |c| = k.

Consider the previous algorithm. We know there is an optimal value for r. Let’s use that r. Each s that is chosen in the new algorithm would be a valid s in the previous algorithm. This holds until we eliminate all points in S, therefore we produce a cover (C, 2r).

Credit to Kevin Canini for the notes!

M₁: 2, 2, 2
M₂: 3, 4
M₃: 6

We can write this as a decision problem. Given the number of machines and a list of job times, and a bound k, is there a way to assign jobs so that Makespan <= k? Makespan is NP-complete, by reduction from Subset Sum! Given a target W and w₁, w₂, … , w_n, split into sets W and S – W where S = sum(w_i). Take |W – (S – W)| = |2W – S| = W₀, a new constant in the list which forces the best solution where both subsets have the same sum = max(W, S – W).

Given a Subset Sum problem, create W₀ = |S – 2W| and let the w’s be the times of jobs. Ask if Makespan works on 2 machines of size <= (S + W₀) / 2.

Claim: “There is a solution to Subset Sum if and only if there is a solution to Makespan.”

Assume a Subset Sum solution. Split the w’s into subsets that sum to W and S – W. W₀ was chosen so we can make them equal, so there exists a Makespan = (S + W₀) / 2, and some machine has a subset of jobs summing to W.

Greedy Strategy: Assign each job to least busy machine, incrementally. How good is this? Call the best Makespan possible T*. The sum(t_i) = total time to be spread over m machines. Some machine must have more than sum(t_i) / m, so T* >= sum(t_i) / m. Consider M_i, the machine with worst time; it’s load is T = T_i. In general, T_j = load on M_j. Our makespan answer is T₁ by the greedy algorithm.

Let t_j be the last time assigned to M_i. At that moment, M_i’s load is T_i – t_j. That must be the smallest load, because we chose it for the last job. So T_i – t_j <= T_k for all k.

For each machine: M(T_i – t_j) <= sum(T_k) = sum(t_j). So T_i – T_j < = sum(t_j) / m &;t= T*. T_i &lt= T* + t_j so T = T_i <= T* + t_j and T* >= worst t_k >= t_j, so T <= T* + T* -> T <= 2 T*.

The greedy algorithm is always within twice the optimal value. If we presort and do the largest jobs first, we can show that T/T* <= 3/2 and T/T* <= 4/3.

Credit to Kevin Canini for the notes!

Imagine what happens when you run Independent Set on a Tree:

The nodes of the indepent set are labelled in red. How do we get an independent set for a tree? Simply, choosing the leaves first is optimal:

Let T = (V, E) be a tree and v be a leaf vertex. I claim that there is a maximum size independent set that contains v. Consider a max size independent set S. If v is in S, we are done. Assume v is not in S. Let u be a parent of v. If u is not in S, we can put v in S instead to get a larger independent set. If u is in S, we can take out u and put v into S to get S’, an independent set of same size = max size.

Here is an algorithm:

Place all leaves into S
while S is not empty {
    Color the leaves
    Delete leaves and parents
}

You can implement this in linear time using DFS–and even if your graph is not a tree, it still may have some tree-like pieces.

Sample NP-Complete Reductions:

Given n processes and m resources:

a) Each process needs several resources at once. Is there a way for k processes to be active at the same time?

Some ideas include Set Packing, Independent Set. To do it with Set Packing, you would ask if there are k subsets of processes that don’t intersect over resources.

b) Special case of (a) where k = 2.

c) Special case of (a) where there are 2 types of resources and processes require at most one of each type.

d) Each resource is requested by at most 2 processes.

Given T = c1 ^ c2 ^ … ^ ck where each clauses c_i is an OR-clause with 3 terms (an instance of 3-SAT) , does there exist and x1 for all x2 does there exist an x3 for all x4 … for all xn-1 such that T is true? This is called quantified 3-SAT (QSAT), an is a kind of alternation thatsshows up in many game AI questions. I.e., is there a move for me such that for all of my opponents’ next move there is another good move for me such that … etc.

One idea for QSAT–try all truth assignments. Define: p^w = polynomial time algorithm given an “oracle” for w. What if you need a solution to an NP-complete problem? Sometimes there are special characteristics of a problem that allows a solution. Sometimes we can define a provably good approximation solution!

Look at vertex cover. Given G, k, is there a vertex cover of size <= k? So, there are n choose k possible vertex covers. We could try them all, but… that’s a lot of options. To check a vertex cover, try each edge e, see if at least one endpoint is in the cover, and repeat. This takes O(n²) time.

Claim: “If G has n nodes, with max-degree d, and if there is a vertex cover of size k, then G has at most k*d edges.”

k*d is better than O(n²). Let s be the vertex cover with |s| = k. There are at most d edges from each vertex in S, there are at most k*d places where S touches some edge either once or twice. |E| <= # touches <= k*d, so trying all the possibilities is no longer O(n choose k * k * (n-1)) but rather O(n^k+1 * k).

Observe that any graph with more than k*(n-1) edges cannot have a vertex cover of size k, and if there is a vertex cover of size k and we completely remove a vertex from the cover, then the remaining graph has a vertex cover of size k-1.

Find(G, k):
    if G has no edges return false
    if G has more than k*(n-1) edges return false
    choose e = (u, v)
        s1 = find(G – {u}, k-1)
        s2 = find(G – {v}, k-1)
        if s1 and s2 fail, then return false
        if(s1) return s1 + {u}
        if(s2) return s2 + {v}

This algorithm is O(k*n*2^k+1). The proof is left as an exercise to the reader.

if a(s) returns “yes” then return “no” otherwise return “yes”

Let y be in NP. What about y’? Let y’ = {s | s not in y}. To say “y is in NP” means that there exists a polynomial time certifier b(*,*): s is in y if and only if there exists t such that b(s, t) returns “yes,” in polynomial bounded size. So, let’s look at y’:

s in y’
<=> s not in y
<=> ! there exists t such that b(s,t) returns “yes”
<=> for all t, b(s, t) returns “no”

Co-NP = {y’ | y is in NP}. Does NP = Co-NP? We don’t know. If we assume P=NP, though, it follows that P=Co-NP as well. Generally the consensus is that NP does not equal Co-NP. Here is a diagram that might help explain P, NP, Co-NP, and PSPACE:

P-SPACE is defined as {x | x has an algorithm using at most polynomial space}. P is a subset of PSPACE, because you cannot use more than polynomial space in only polynomial time. To say x is in PSPACE implies that x’ is in PSPACE using the same inversion algorithm as above. You can also create the following chain:

x in Co-NP => x’ in NP => x’ in SPACE => x in PSPACE

Given distances between n cities and a bound D, is there a tour (visit all cities and return home) of cost at most D? Note that using TSP to prove NP-completeness of other problems is quite hard.

Claim: “TSP is NP-complete”

Proof: TSP is in the set NP. Given a valid tour, we can check that it is valid and has a cost &lt= D in polynomial time. Then, we want to show that Hamiltonian Cycle reduces to TSP in polynomial time. Given an HC problem as a graph G, we can show how to build the travelling salesman problem. Using the same vertices, place an edge between each pair with cost 1 if the edge exists in G, otherwise place the edge with cost 2.

Claim: “G has a Hamiltonian Cycle if and only if G’ has a TSP tour of cost &lt= n (vertices)”

Proof (=>): Assume G has a HC. Then the cycle in G’ is a tour and its cost is <= n.

Proof (<=): Assume G’ has a TSP tour of cost <= n. Every node is visited, and each edge has cost 1. The corresponding cycle is a valid HC.

3D Matching:

Given disjoint sets X, Y, and Z each of size n and given a set of triples T is a subset of (X, Y, Z) is there a subset of T of size n that covers all X + Y + Z? We can show that 3-SAT reduces to 3D Matching in polynomial time, so 3D matching is NP-complete.

Graph Coloring:

A graph g is said to have a k-coloring if there is a way to assign colors to vertices such that all adjacent verticex do not share the same color. Given a graph g and a bound k, is G k-colorable? Graph coloring is NP-complete, but on planar graphs 2-coloring has a simple DFS algorithm, and >=4-coloring simply returns “yes.”

Subset Sum:

Given w₁, … , w_n in {Naturals} and a target w_i is there a subset of w₁, … , w_n whose sum is exactly w_i?

NP-Complete “Types”:

All the NP Complete problems we have covered so far can be grouped by similarity into the following categories:

• Packing problems: Set Packing, Independent Set
• Covering problems: Vertex & Set Cover
• Partitioning problems: 3D Matching, Graph coloring
• Sequencing problems: Hamiltonian Cycle, Hamiltonian Path, Travelling Salesman Problem
• Numerical problems: Subset sum
• Constraint problems: SAT, 3-SAT, and Circuit-SAT