slides - SNAP - Stanford University

CS 322: (Social and Information) Network Analysis

Jure Leskovec

**Stanford** **University**

How do networks evolve at the macroscopic

level?

Are there global phenomena of network

evolution?

What are analogs of the “small small world” world and

“power‐law degrees”

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

What is the relation between the number

of nodes n(t) and number of edges e(t)

over time t?

How does diameter change as the network

grows?

How does degree distribution evolve as

the network grows?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

N(t) … nodes at time t

E(t) … edges at time t

Suppose that

N(t+1) = 2 * N(t)

Q: what is

E(t+1) =

AA: over‐doubled! d bl d!

But obeying the Densification Power Law

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. KDD 05]

Citations among

physics papers

1992:

1,293 papers,

2,717 citations

2003 2003:

29,555 papers,

352,807 citations

For each month

M, create a graph

of all citations up

to month M

E(t)

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. KDD 05]

1.69

N(t)

Densification Power Law

[Leskovec et al. KDD 05]

the number of edges grows faster than the

number of nodes – average g degree g is increasing g

or

equivalently

a … densification exponent: 1 ≤ a ≤ 2:

a=1: linear growth – constant out‐degree

(traditionally assumed)

a=2: quadratic growth – clique

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Prior work on Power Law graphs hints at

Slowly growing diameter:

diameter ~ O(log N)

diameter ~ O(log log N)

Wh What is i happening h i in i real l data? d ?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. KDD 05]

Citations among

physics papers

1992 –2003

One graph per

year

diameter

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. KDD 05]

time i [ [years] ]

Is shrinking

diameter just j a

consequence of

densification?

diammeter

[Leskovec et al. TKDD 07]

Erdos‐Renyi

random graph

Densification

exponent p a =1.3

size of the graph

Densifying random graph has increasing

diameterThere diameterThere is more to shrinking diameter

than just densification

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Is it the degree sequence?

Compare diameter of a:

True network (red)

Random network with

diameeter

[Leskovec et al. TKDD 07]

Citations Cit ti

the same degree

distribution (blue) size of the graph

Densification + degree sequence

give shrinking h k diameter d

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. TKDD 07]

How does degree distribution evolve to allow

for densification?

2 Options:

1) Degree exponent is constant:

Fact 1: For degree exponent 1

[Leskovec et al. TKDD 07]

How does degree distribution evolve to allow

for densification?

2 Options:

2) Exponent n evolves with graph size n:

Fact 2:

Citation network

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. TKDD 07]

Densification and Shrinking diameter can be

observed in many real life graphs:

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

What explains the Densification Power Law

and Shrinking diameters

Can we find a simple model of local behavior,

which naturally leads to the observed

phenomena?

Yes! Community Guided Attachment

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Let Let’s s assume the

community

**University**

[Leskovec et al. TKDD 07]

structure Science Arts

One expects many

within‐group

fi friendships dhi and d

fewer cross‐group

ones

How hard is it to

cross communities? ii ?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

CS Math Drama Music

Self‐similar university

community it structure t t

Assume the cross‐community cross community linking

probability of nodes at tree‐distance h is

scale‐free scale free

Then the cross‐community linking

probability p yis:

where: c ≥ 1 … the Difficulty ff y constant

h … tree‐distance

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

n=2k n = 2 nodes reside in the leaves of the b‐way b way

community hierarchy (assume b=2)

Each node then independently creates edges

based the community hierarchy, hierarchy

How many edges m are in a graph of n nodes?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Claim: l Community Guided ddAttachment h

random graph model, the expected out‐

ddegree of f a node d is i proportional i lto

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. TKDD 07]

What is expected out‐degree out degree of a node x?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Claim: The Community Guided Attachment

leads to Densification Power Law with

exponent p

a … ddensification ifi ti exponent t

b … community tree branching factor

c … difficulty constant constant, 1 ≤ c ≤ b

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

DPL:

Gives any non‐integer Densification

exponent

If c=1: c = 1: easy to cross communities

Then: a=2, quadratic growth of edges – near

clique

If c = b: hard to cross communities

Then: a=1 a=1, linear growth of edges – constant

out‐degree

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. TKDD 07]

Community tree evolves by a complete new

level of nodes being added in each time step

Now nodes also reside in non‐leafs non leafs of the tree

Can show:

Densification + power‐law degree distribution

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Topics of web pages:

Each node represents k pages

Nodes everywhere in the tree

Link up h levels with prob. ~ -h

[Leskovec et al. TKDD 07]

Simplification: only bottom level creates links

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Expected degree of a node at height h:

Fraction of nodes with degree >d as a

function of d?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Leskovec et al. TKDD 07]

Normal N(x) =

Limit of avg avg. of indep indep. draws from a random var

from any distribution

X is lognormally distributed if Y = ln X is

normally distributed

Log‐normal Log normal density

f(x)=

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

f

( x)

Properties:

1

2 2

1

x

e

(ln

x

)

2

/ 2

Finite mean/variance.

Skewed: mean > median > mode

Multiplicative:

X 1 lognormal, X 2 lognormal implies X 1X 2 lognormal.

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

2

[Mitzenmacher]

Looks similar to ppower‐law on log‐log g g pplot:

Power‐law has linear log‐density

ln f ( x )

ln x

ln c

For large , lognormal has nearly linear log‐

density:

lln

f

( x )

lln

x

lln

2

2 ln x

[Mitzenmacher]

Similarly Similarly, both have near linear log log‐ccdfs. ccdfs

Question: how to differentiate them empirically?

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

2

2

Question: Is this distribution lognormal or a

power law?

Reasonable follow follow‐up: up: Does it matter?

Generative model:

Size‐independent + growth

Size (e (e.g., g popularity, popularity degree) start at one and

changes in each step by a random multiplicative

factor F

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Start with an organism of size X 0.

At each time step, size changes by a random

multiplicative factor factor.

X Xt

F Ft

1

X Xt

1

Claim: If F Ft is taken from a lognormal

distribution, each Xt is lognormal.

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Claim: If F t is taken from a lognormal

distribution, each Xt is lognormal.

If F t are independent, identically distributed

then (by CLT) X Xt converges to lognormal

distribution.

ln X t = size at time t = ln F 1+ ln F 2+…+ln F t

ln X t = ln F j ~ normal dist X t is lognormal

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

If there exists a lower bound:

X t max( ,

F t t

1 1X t t

1 1)

[Mitzenmacher]

then Xt converges to a power law distribution

[Champernowne [Champernowne, 1953]

Lognormal model easily pushed to a power

law model

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Consider continuous version of lognormal

generative model:

At time t, log X t is normal with mean t t and

variance 2 t

Suppose observation time is distributed

exponentially

E.g., When Web size doubles every year.

RResulting lti distribution di t ib ti is i Double D bl Pareto. P t

[Mitzenmacher]

Between lognormal and Pareto.

Li Linear tail tilon a log‐log l l chart, h t but b ta lognormal l lb body. d

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

[Mitzenmacher]

Hierarchical model that produces power‐laws power laws

Zipf’s Zipf s law – how does the frequency of a word

depend on its rank?

Empirically: rank ~1/freq

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Mandelbrot experiment: design a language

over a d‐ary alphabet to optimize information

per character.

Probability of j th most frequently used word is p j.

Length of j th most frequently used word is c j.

AAverage iinformation f ti per word: d

H

p

j j log2

p

Average characters per word:

C

j

p jc

Optimization leads to power law.

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

j

j

Alphabet of c letters plus spacebar

Monkey sits at a typewriter and randomly

pushes keys

Uniformly random sequence of c+1 symbols –

spaces mark word boundaries

spaces mark word boundaries

Word ranks

11, 2, 2 …, c 11‐letter letter words

c+1, …, c 2 +c+1 2‐letter words

c2 c +c+1+1 3 letter words

2 +c+1+1, … 3‐letter words

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Each j letter word has frequency (c+1) ‐(j+1)

Each j‐letter word has frequency (c+1) (j )

Pick representative point c j for j‐letter word

Rank cj is a j‐letter word with freq (c+1) ‐(j+1)

Rank cj is a j‐letter word with freq (c+1) (j )

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis

Write (c+1) ‐(j+1) Write (c+1) as a function of rank

(j ) as a function of rank

10/15/2009 Jure Leskovec, **Stanford** CS322: Network Analysis