1 - Framingham State University

1. Sets, languages, and models 

David M Keil, Framingham State University 

CSCI 460 Theory of Computing 

1. Sets, languages, 

and models 

1. Countable sets and formal languages 

2. Uncountable sets 

3. Streams and coinduction 

4. Circuit and lookup computation 

David Keil Theory of Computing 12/12 1 

Inquiry 

• Are the sets of natural numbers and 

strings of comparable size? 

• Are all infinite sets the same size? 

• Can we define infinite sets of infinitely 

large data streams? 


David Keil Theory of computing 6/12


Objectives 

1a. Use the notation of formal language 

theory* 

1b. Describe the logic-circuit model of 

computation* 

1c. Prove that a set is countable 

1d. Prove that a set is uncountable 

1e. Explain or write a coinductive definition 


How Topic 1 relates to computing 

• We show that some problems are 

algorithmically unsolvable 

• We show a diagonal proof method that is 

used again in topic 4 

• Computation deals with finite, infinite, and 

uncountable objects or sets 

• Computations are operations on sets of 

strings (languages) 




• How big is a set? 

• What does countability have to do with 

computing? 

• How many of the following exist? 

– natural numbers 

– rational numbers 

– real numbers 

– functions 

– programs 

– sets 


David Keil Theory of computing 6/12 

1. Countable sets and formal languages 

Countability 

• The natural numbers are countable, i.e., 

they can be arranged in a particular order 

• A bijection is a one-to-one correspondence, 

i.e., a relation between two sets that is both 

surjective and injective 

• A set is countable iff there exists a bijection 

between it and the natural numbers (ℕ) 

• Countable sets tend to be sets of finite 

objects, defined inductively 

David Keil Theory of Computing 12/12 6



Peano’s axioms: definition by induction 

1. 0 is a natural number (0 N) 

2. Every natural number n has a unique 

successor, n, also a natural number 

(n N) n N 

3. All natural numbers follow (1) or (2) 

(n N) n = 0 (m N) n = m 

• Significance: These axioms, or assumptions, 

provide a formal logical basis to work with 

counting numbers. Note: (2) is recursive 

• Computation is a formal way to manipulate 

numbers and objects representable by them. 

David Keil Theory of Computing 12/12 

Theorem: ℕ is infinite 

Proof by contradiction: 

1. Suppose ℕ is finite 

2. Then let n be the largest element of ℕ 

3. Consider the number (n + 1). It cannot be in ℕ, 

because it is larger than ℕ’s largest element 

4. But by definition of ℕ, any successor of an 

element of ℕ is in ℕ. 

5. Hence by the contradiction of (3) and (4), we 

must reject (1). 

6. Hence ℕ is infinite. 


7


Strings 

• Alphabet: a finite set of symbols, e.g., 

{0,1}, 

{0, 1, …9}, {a, b, c, …, z} 

• A string is a sequence of symbols over a 

finite alphabet 

• Notation: 

– x R is the string x reversed 

– n 0(x) is the number of zeroes in string x 



The set of strings over an alphabet 

• * is the set of strings over finite alphabet 

• Recursive definition: 

– Base: * ( = = null string) 

– Recursive: a s * sa * 

– Restriction: only objects defined as above 

are strings 

• In set notation, 

* = {} { ax | a , x * } 



Str(n) = 

A bijection ℕ {0,1}* 

if n = 0 

Str(n/2) + ‘0’ if Odd(n) n > 0 

Str(n/2) + ‘1’ otherwise 

Num(s) = 

0 if s = 

2 Num(s[1 .. |s| - 1]) + 

s[|s|] + 1 otherwise 

• This bijection is the core of a proof that 

{0,1}* is countable 


Cardinalities of sets 

• Informally, their sizes 

• For infinite sets, we say that two sets have 

the same cardinality iff there is a bijection 

(1-to-1 correspondence) between their 

elements 

• Example: Card (EVEN) = Card (ℕ) 

Proof: EVEN 0 = 0, EVEN 1 = 2, etc. 

• Example: Card (ℚ) = Card (ℕ) 

Proof: Zig-zag through a table of rationals 






Formal languages 

• Language: a set of strings over an alphabet 

• Languages may be enumerated: 

, 0, 1, 00, 01, 10, 11, … 

• * is the set of all strings over 

• For any language L over , L * 

• Inductively, * = {xy | x y * } 

Defining languages 

• 0 n 1 n is the language of strings with n zeroes 

followed by n ones 

• Let alphabet be {0,1}, let be the null 

string 

• Then 0 = {}, 1 = {(0), (1)} 

2 = {00, 01, 10, 11} 

k = the set of strings of length k 

* = U k N k 



Operations on languages 

• Concatenation (L 1 L 2 ): strings that are a 

string in L 1 followed by one in L 2 

• L 1 L 2 = {xy | x L 1 y L 2} 

• Union (selection, L 1 L 2, L 1 | L 2): 

strings either in L 1 or L 2 

L 1 L 2 = {x | x L 1 x L 2} 

• Intersection (L 1 L 2) strings that are in 

both L 1 and L 2 


Kleene star languages 

• Kleene star (Iteration, L*): strings that are 

sequences of strings in L 

L * = {xy | x L y L * } 


• Alternative definition: 

– L 0 = {}, L 1 = L, L 2 = LL, L 3 = LLL, etc. 

– LL ={xx | x L} 

– L* = U k N L k 

– Example: If L = {0, 00} then 

LL = {00, 000, 0000} 



Countability of strings 

• To help prove equivalence of number-based 

and string-based models of computation, we 

show mappings of any string to a natural 

number and any natural number to a string 

• Enumeration according to bit representation: 

N {0,1}* 

0 

1 0 

2 1 

3 00 

4 01 


Languages and countability 

• Computation operates on strings; 

instances of models of computation 

accept sets of strings 

• A language may be represented as an 

infinite bit sequence, with a 1 bit in a 

certain bit location denoting a string’s 

membership in the language: 

{0, 01, 11, 100} = 10101100… 





Countable set: the rational numbers 

• The rationals may 

be enumerated as 

at right, where the 

numerator is the 

row number and 

the denominator is 

the column number 

• Reference: “The set of rational numbers is countable,” 

(http://www.homeschoolmath.net/teaching/rationalnumbers-countable.php) 


2. Uncountable sets 

• Are all infinite sets the same size? 

• Does infinity matter in the real 

finite world? 



Uncountability 

• We show that some sets, such as the 

real numbers, are not countable 

• We prove this by showing that no 

bijection between such a set and the set 

of natural numbers can exist 

• This proof and its method imply limits 

on the power of algorithmic 

computation 


The set of real numbers 

• Intuitively, the real numbers express analog or 

continuous quantities; they give all the 

possible weights or distances. 

• We may consider the size of the set ℝ, of reals, 

as corresponding to all the points on a line 

segment, line, plane, space, or space/time 

• Another definition: The reals are all the 

numbers that can be expressed using an 

infinite sequence of digits after the decimal or 

binary point 




Theorem: |ℝ| > |ℕ| 

Proof (Cantor) : 

1.Suppose ℝ were countable 

2.Then the interval (0, 1] could be enumerated as: 

r 1 = 0 . b 1,1 b 1,2 b 1,3 … 

r 2 = 0 . b 2,1 b 2,2 b 2,3 … 

… where b m,n {0,1} 

3.Now, consider the real number s = 

0. b 1,1 b 2,2 b 3,3 … 

That is, for all n, bit n is b n,n (bitwise negation 

of the diagonal of r) 


Cantor’s proof (cont’d) 

4. Now, since for every i, the i th bit of s is 

different from the i th bit of r i, so s differs 

from every element of r, so there is no 

element of sequence r that matches s 

5. Hence r does not contain s, which is 

clearly a real number 

6. Hence we have a contradiction and must 

reject the supposition 

7. Hence ℝ is not countable 




Cantor’s proof, illustrated 

Supposed enumeration of ℝ: 

r 1 = (0, 0, 0, 0, 0, 0, 0, ...) 

r 2 = (1, 1, 1, 1, 1, 1, 1, ...) 

r 3 = (0, 1, 0, 1, 0, 1, 0, ...) 

r 4 = (1, 0, 1, 0, 1, 0, 1, ...) 

r 5 = (1, 1, 0, 1, 0, 1, 1, ...) 

r 6 = (0, 0, 1, 1, 0, 1, 1, ...) 

r 7 = (1, 0, 0, 0, 1, 0, 0, ...) 

... 



Real number that differs 

with every element in 

enumeration in at least 

one bit: 

s = (1, 0, 1, 1, 1, 0, 1, ...) 

Summary of Cantor’s proof 

• “The proof shows that no matter how a list 

of real numbers were arranged, a real 

number could still be defined that would not 

be in the list” 

T. DiRienzo, J. Bartlett, 2/09 

• A diagonal proof is both by construction and 

by contradiction 



The predicates are uncountable 


• Start with the set of functions 

{ f : {0} {0,1}} 

• This is a set of two functions: 

{{(0,1)}, {(0,0)}} 

• The set { f : {0, 1} {0,1}} has four elements, 

{ f : {0, 1, 2} {0,1}} has eight, etc. 

• There are 2 |ℕ| predicates f : ℕ {0,1} 

• Question: Is this the same as, or more than, the 

cardinality of the natural numbers? 

• Compare with the cardinality of reals 


Can all predicates on ℕ be 

computed in Java? 

1. Enumerate all Java methods that take an integer 

parameter and return 0 or 1, as J 1, J 2, J 3, … in a 

bitwise ordering 

2. Consider the predicate f : ℕ {0,1} s.t. f (n) = 1 

if J n(n) = 0 or J n(n) hangs, f (n) = 0 if J n(n) = 1 

3. The predicate f differs in its behavior from each 

element of our enumeration of Java methods 

4. Hence f is a predicate not computed by any Java 

method 



Results 

• ℕ is countable (cardinality 0, aleph-null) 

• ℝ is uncountable (cardinality 1) 

• The set of Java methods is countable using 

their bit representations 

• The predicates f : ℕ {0,1} are uncountable 

• There are 2 0 = 1 predicates on ℕ 

• The sets of natural numbers are uncountable 

(see handout) 


3. Streams and coinduction 

• Can we define infinite sets of infinitely 

large data streams? 

• How can infinite I/O be described 

mathematically? 

• What is induction without a base case? 



30


Environment 

Streams 

Interaction stream 

inputs 

outputs 

• A reactive system may receive an infinite 

stream of inputs and may emit the 

corresponding infinite stream of outputs 


Program 

• Sets of streams express interactive behavior 

• Coinduction defines sets of infinite objects 

as induction defines sets of finite ones 


Inductively defined sets 

• (i) 0 is a natural number; 

(ii) Every n ℕ has a unique successor, n; 

(iii) i and ii are the only ways to obtain a 

natural number (Peano) 

• Set of expressions using numerals, +, and ( ): 

expression 

numeral | 

numeral + expression | 

( expression ) 

• The definition of such a set may use itself 

• But such a set does not contain itself 


32


Inductive and coinductive 

definitions of languages 

• * = {} U {ax | a , x * } 

(note base case ) 

• “L is a language over alphabet *” 

means that L * 

• A stream is an infinite string 

• A stream language is a set of streams 

• Example, defined coinductively: 

= {ax | a , x } 

(note lack of base case) 


Well-founded sets 

• (B. Russell) Let the village hair stylist be 

the person who cuts the hair of everyone 

who doesn’t cut own hair 

• Question: Who cuts the hair-stylist’s hair? 

• Similar Questions: Is there a set of all sets? 

If so, does it contain itself ? Is there a set of 

all sets that don't contain themselves? 

• In classical (well founded) set theory, it is 

meaningless to say a set belongs or doesn't 

belong to itself (Foundation Axiom) 



34


Another diagonal proof 

• Theorem: The notion “set of all sets” leads 

to a contradiction 

• Proof: Consider = { A : A A } 

(sets not members of themselves) 

• ( ) ( ) , 

( ) ( ) , contradictions 

• Note role of “Spoiler” instance 

• We say that , are not well founded 

(NWF) 



Streams and non-well-founded sets 

• A non-well-founded set may contain itself, 

directly or indirectly 

• Example: A = { B, C }; B = { A, D } 

• Every stream of characters is a character, 

followed by a stream of characters 

• Streams contain streams, which contain 

streams, which contain streams, ... 

• Ex.: The set of all bit streams {0,1} consists 

of any infinite sequence that consists of 0 or 1 

followed by a stream 


36


Streams and coinduction 


• Induction defines countable sets of finite 

objects: * = {} { ax | a , x * } 

• Coinduction 

– is a dual of induction (recursion), lacking a 

base case 

– is used to define sets of infinite objects 

• The set of streams over an alphabet, expressing 

ongoing processes such as interaction: 

= { ax | a , x } 


Least and greatest fixed points 

• A fixed point x of function f is a value x 

(which may be a set), for which f (x) = x 

• Whereas induction is characterized by 

minimality conditions (least fixed points), 

coinduction has a maximality condition 

(greatest fixed point) 

• Minimality example: + is the smallest set that 

fits the spec {ax | a , x *} 

• Maximality example: is the largest set that 

fits the spec {ax | a , x } 


37


4. Circuit and lookup computation 

• What is a function on a finite set? 

• What is a property of a finite set? 

• How can a function on a finite domain be 

defined? 

• What’s a way to compute such a function? 

• What’s a simple model of simple 

computation? 


Problems and languages 

• A computational problem with a Boolean 

(yes/no) solution is called a decision problem 

• Computing devices that solve them are called 

acceptors or recognizers 

• A decision problem corresponds to a 

language: all the strings that get a “yes” 

decision 

• Other problems, with string solutions, are 

called transduction problems 




Decision and transduction 

• A decision problem requires yes/no responses 

to input 

• Hence it is also known as a languagerecognition 

problem 

• A transduction computation produces a string 

as output 

• Hence its solution is the computation of a 

function 

• We speak of decidable languages (problems) 

or computable functions 



Some language-decision problems 

• Some languages: 

– 1 | 0 {0, 1} 

– (1 | 0) 1 {11, 01} 

– 1* {, 1, 11, …} 

– 1*0 {0, 10, 110, 1110, …} 

– 0(1|0)* {0, 00, 01, 000, 001, 

010, 011, …} 

• Q: How do we build a simple machine to 

accept strings in one of these languages? 


41


No-loop computation 

• The simplest type of computation uses no 

loops, but maps from finite domains 

• Examples: lookup table; logic circuit 


• Other equivalent models: 

– Finite computation trees 

– Flowcharts with bounded input, no loops 

– Formulas in propositional logic 

– Finite languages L k for any or k 


Logic circuits 

• A model for the computation 

of functions f : {0,1} m {0,1} n 

where m is the arity of f 

(number of inputs to circuit) 

and n is the number of outputs 

• Proposition: for any such function f, a logic 

circuit that computes f may be constructed from a 

finite number of , , or gates 

• A logic circuit computes a function on strings of 

bounded size, i.e., functions with a finite domain 



Lookup tables 

• A function on a finite domain may be 

represented and effectively computed 

using a lookup table 

Square function 



a b a xor b 

0 0 0 

0 1 1 

1 0 1 

1 1 0 

XOR operator 

Labeled transition systems 

• Definition: a digraph, where vertices denote 

state, labeled edges denote transitions 

• Example (right): an input string 

of bits is read from the 

top, following edges 

• A loopless transition 

system can accept 

any finite language of strings that leave the 

system in an accepting state 



Transducers and accepters 

• Theorem:Any computation of a function 

f : {0, 1} * {0, 1} n can be done by n accepters 

• Proof: 

1. for i n, let L i = {x | bit i of f(x) is 1} 

2. Construct accepters M 1 .. i for L 1 .. i as follows. 

3. M i decides the ith bit of f (x) for any string x. 

4. Let P i be the predicate computed by M i 

5. So f (x) = Concat i m(P i(x)) 

6. Hence f (x) is computable by M 1..n 


Barwise-Moss. Vicious Circles, 1996 

S. Epp. Discrete Mathematics with 

Applications. Brooks/Cole, 2011. 

Finsler, 1920s (NWF sets) 

Goldin-Wegner (at www.engr.uconn.edu/~dqg) 

Peano, 1889 (induction) 

J. Savage. 

References

1 - Framingham State University

Create successful ePaper yourself

Delete template?

Save as template?