8/19/16. Clustering. Clustering is a hard problem. Clustering is a hard problem

Transcription

1 Clusterng Patrce Koehl Department of Bologcal Scences Natonal Unversty of Sngapore Clusterng s a hard problem Many possbltes; What s best clusterng? Clusterng s a hard problem 2 clusters: easy 1

2 Clusterng s a hard problem 4 clusters: dffcult?? Many possbltes; What s best clusterng? Clusterng Ø Herarchcal clusterng Ø K-means clusterng Ø How many clusters? Clusterng Ø Herarchcal clusterng Ø K-means clusterng Ø How many clusters? 2

3 Herarchcal Clusterng To cluster a set of data D={P 1, P 2,,P N }, herarchcal clusterng proceeds through a seres of parttons that runs from a sngle cluster contanng all data ponts, to N clusters, each contanng 1 data ponts. Two forms of herarchcal clusterng: Agglomeratve a, b, c, d, e c, d, e a, b d, e a b c d e Dvsve Agglomeratve herarchcal clusterng technques Ø Starts wth N ndependent clusters: {P 1 }, {P 2 },,{P N } Ø Ø Fnd the two closest (most smlar clusters, and on them Repeat step 2 untl all ponts belong to the same cluster Methods dffer n ther defnton of nter-cluster dstance (or smlarty Agglomeratve herarchcal clusterng technques 1 Sngle lnkage clusterng Cluster A Dstance between closest pars of ponts: { P, P, P A P B} A, B = mn, Cluster B 2 Complete lnkage clusterng Cluster A Dstance between farthest pars of ponts: { P, P, P A P B} A, B = max, Cluster B 3

4 Agglomeratve herarchcal clusterng technques 3 Average lnkage clusterng Cluster A: N A elements Mean dstance of all mxed pars of ponts: P, P P AP B A, B = N AN B 4 Average group lnkage clusterng Mean dstance of all pars of ponts A, B = P T P T P, P N 2 T Cluster A Cluster B: N B elements Cluster B Cluster T Clusterng Ø Herarchcal clusterng Ø K-means clusterng Ø How many clusters? K-means clusterng ( 4

5 K-means clusterng ( K-means clusterng ( K-means clusterng ( 5

6 K-means clusterng ( Clusterng Ø Herarchcal clusterng Ø K-means clusterng Ø How many clusters? Cluster valdaton Clusterng s hard: t s an unsupervsed learnng technque. Once a Clusterng has been obtaned, t s mportant to assess ts valdty! The questons to answer: Ø Dd we choose the rght number of clusters? Ø Are the clusters compact? Ø Are the clusters well separated? To answer these questons, we need a quanttatve measure of the cluster szes: Ø ntra-cluster sze Ø Inter-cluster dstances 6

7 Inter cluster sze Several optons: Ø Sngle lnkage Ø Complete lnkage Ø Average lnkage Ø Average group lnkage Intra cluster sze Several optons: - Complete dameter: - Average dameter: - Centrod dameter: For a cluster S, wth N members and center C: ( Δ(S = max (x,y S 2 x, y 1 Δ(S = N(N 1 Δ(S = 2 N x S (x,y S x y x,c x, y 1 Dunn s ndex Cluster Qualty For a clusterng wth K clusters: - & ( δ(s,s * 0 D = mn/ ( mn' 1 K 1 K max( Δ(S + 2 /. ( k 1 k K,( 2 1 Large values of D correspond to good clusters 2 Daves-Bouldn s ndex DB = 1 % K max Δ(S + Δ(S ( ' & δ(s,s * Low values of DB correspond to good clusters 7

8 Cluster Qualty: Slhouette ndex Defne a qualty ndex for each pont n the orgnal dataset: Ø For the th obect, calculate ts average dstance to all other obects n ts cluster. Call ths value a. Ø For the th obect and any cluster not contanng the obect, calculate the obect s average dstance to all the obects n the gven cluster. Fnd the mnmum such value wth respect to all clusters; call ths value b. Ø For the th obect, the slhouette coeffcent s s( = b( a( max a(,b( ( Cluster Qualty: Slhouette ndex Note that: 1 s( 1 Ø s( = 1, s lkely to be well classfed Ø s( = -1, s lkely to be ncorrectly classfed Ø s( = 0, ndfferent Cluster Qualty: Slhouette ndex Cluster slhouette ndex: S(X = 1 N N =1 s( Global slhouette ndex: GS = 1 S(X K =1 Large values of GS correspond to good clusters N 8