Package weka.clusterers
Class EM
- All Implemented Interfaces:
Serializable,Cloneable,Clusterer,DensityBasedClusterer,NumberOfClustersRequestable,CapabilitiesHandler,CapabilitiesIgnorer,CommandlineRunnable,OptionHandler,Randomizable,RevisionHandler,WeightedInstancesHandler
public class EM
extends RandomizableDensityBasedClusterer
implements NumberOfClustersRequestable, WeightedInstancesHandler
Simple EM (expectation maximisation) class.
EM assigns a probability distribution to each instance which indicates the probability of it belonging to each of the clusters. EM can decide how many clusters to create by cross validation, or you may specify apriori how many clusters to generate.
The cross validation performed to determine the number of clusters is done in the following steps:
1. the number of clusters is set to 1
2. the training set is split randomly into 10 folds.
3. EM is performed 10 times using the 10 folds the usual CV way.
4. the loglikelihood is averaged over all 10 results.
5. if loglikelihood has increased the number of clusters is increased by 1 and the program continues at step 2.
The number of folds is fixed to 10, as long as the number of instances in the training set is not smaller 10. If this is the case the number of folds is set equal to the number of instances.
Missing values are globally replaced with ReplaceMissingValues. Valid options are:
EM assigns a probability distribution to each instance which indicates the probability of it belonging to each of the clusters. EM can decide how many clusters to create by cross validation, or you may specify apriori how many clusters to generate.
The cross validation performed to determine the number of clusters is done in the following steps:
1. the number of clusters is set to 1
2. the training set is split randomly into 10 folds.
3. EM is performed 10 times using the 10 folds the usual CV way.
4. the loglikelihood is averaged over all 10 results.
5. if loglikelihood has increased the number of clusters is increased by 1 and the program continues at step 2.
The number of folds is fixed to 10, as long as the number of instances in the training set is not smaller 10. If this is the case the number of folds is set equal to the number of instances.
Missing values are globally replaced with ReplaceMissingValues. Valid options are:
-N <num> number of clusters. If omitted or -1 specified, then cross validation is used to select the number of clusters.
-X <num> Number of folds to use when cross-validating to find the best number of clusters.
-K <num> Number of runs of k-means to perform. (default 10)
-max <num> Maximum number of clusters to consider during cross-validation. If omitted or -1 specified, then there is no upper limit on the number of clusters.
-ll-cv <num> Minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters. (default 1e-6)
-I <num> max iterations. (default 100)
-ll-iter <num> Minimum improvement in log likelihood required to perform another iteration of the E and M steps. (default 1e-6)
-V verbose.
-M <num> minimum allowable standard deviation for normal density computation (default 1e-6)
-O Display model in old format (good when there are many clusters)
-num-slots <num> Number of execution slots. (default 1 - i.e. no parallelism)
-S <num> Random number seed. (default 100)
-output-debug-info If set, clusterer is run in debug mode and may output additional info to the console
-do-not-check-capabilities If set, clusterer capabilities are not checked before clusterer is built (use with caution).
- Version:
- $Revision: 15520 $
- Author:
- Mark Hall (mhall@cs.waikato.ac.nz), Eibe Frank (eibe@cs.waikato.ac.nz)
- See Also:
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoidbuildClusterer(Instances data) Generates a clusterer.double[]Returns the cluster priors.Returns the tip text for this propertyReturns the tip text for this propertyReturns default capabilities of the clusterer (i.e., the ones of SimpleKMeans).double[][][]Return the normal distributions for the cluster modelsdouble[]Return the priors for the clustersbooleangetDebug()Get debug modebooleanGet whether to display model output in the old, original format.intGet the maximum number of clusters to consider when cross-validatingintGet the maximum number of iterationsdoubleGet the minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters when cross-validating to find the best number of clustersdoubleGet the minimum improvement in log likelihood necessary to perform another iteration of the E and M steps.doubleGet the minimum allowable standard deviation.intGet the number of clustersintGet the degree of parallelism to use.intGet the number of folds to use when cross-validating to find the best number of clusters.intReturns the number of runs of k-means to perform.String[]Gets the current settings of EM.Returns the revision string.Returns a string describing this clustererReturns an enumeration describing the available options.double[]Computes the log of the conditional density (per cluster) for a given instance.static voidMain method for testing this class.Returns the tip text for this propertyReturns the tip text for this propertyReturns the tip text for this propertyReturns the tip text for this propertyReturns the tip text for this propertyintReturns the number of clusters.Returns the tip text for this propertyReturns the tip text for this propertyReturns the tip text for this propertyReturns the tip text for this propertyvoidsetDebug(boolean v) Set debug mode - verbose outputvoidsetDisplayModelInOldFormat(boolean d) Set whether to display model output in the old, original format.voidsetMaximumNumberOfClusters(int n) Set the maximum number of clusters to consider when cross-validatingvoidsetMaxIterations(int i) Set the maximum number of iterations to performvoidsetMinLogLikelihoodImprovementCV(double min) Set the minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters when cross-validating to find the best number of clustersvoidsetMinLogLikelihoodImprovementIterating(double min) Set the minimum improvement in log likelihood necessary to perform another iteration of the E and M steps.voidsetMinStdDev(double m) Set the minimum value for standard deviation when calculating normal density.voidsetMinStdDevPerAtt(double[] m) voidsetNumClusters(int n) Set the number of clusters (-1 to select by CV).voidsetNumExecutionSlots(int slots) Set the degree of parallelism to use.voidsetNumFolds(int folds) Set the number of folds to use when cross-validating to find the best number of clusters.voidsetNumKMeansRuns(int intValue) Set the number of runs of SimpleKMeans to perform.voidsetOptions(String[] options) Parses a given list of options.toString()Outputs the generated clusters into a string.Methods inherited from class weka.clusterers.RandomizableDensityBasedClusterer
getSeed, seedTipText, setSeedMethods inherited from class weka.clusterers.AbstractDensityBasedClusterer
distributionForInstance, logDensityForInstance, logJointDensitiesForInstance, makeCopiesMethods inherited from class weka.clusterers.AbstractClusterer
clusterInstance, doNotCheckCapabilitiesTipText, forName, getDoNotCheckCapabilities, makeCopies, makeCopy, postExecution, preExecution, run, runClusterer, setDoNotCheckCapabilitiesMethods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, wait, wait, waitMethods inherited from interface weka.clusterers.Clusterer
clusterInstance
-
Constructor Details
-
EM
public EM()Constructor.
-
-
Method Details
-
globalInfo
Returns a string describing this clusterer- Returns:
- a description of the evaluator suitable for displaying in the explorer/experimenter gui
-
listOptions
Returns an enumeration describing the available options.- Specified by:
listOptionsin interfaceOptionHandler- Overrides:
listOptionsin classRandomizableDensityBasedClusterer- Returns:
- an enumeration of all the available options.
-
setOptions
Parses a given list of options. Valid options are:-N <num> number of clusters. If omitted or -1 specified, then cross validation is used to select the number of clusters.
-X <num> Number of folds to use when cross-validating to find the best number of clusters.
-K <num> Number of runs of k-means to perform. (default 10)
-max <num> Maximum number of clusters to consider during cross-validation. If omitted or -1 specified, then there is no upper limit on the number of clusters.
-ll-cv <num> Minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters. (default 1e-6)
-I <num> max iterations. (default 100)
-ll-iter <num> Minimum improvement in log likelihood required to perform another iteration of the E and M steps. (default 1e-6)
-V verbose.
-M <num> minimum allowable standard deviation for normal density computation (default 1e-6)
-O Display model in old format (good when there are many clusters)
-num-slots <num> Number of execution slots. (default 1 - i.e. no parallelism)
-S <num> Random number seed. (default 100)
-output-debug-info If set, clusterer is run in debug mode and may output additional info to the console
-do-not-check-capabilities If set, clusterer capabilities are not checked before clusterer is built (use with caution).
- Specified by:
setOptionsin interfaceOptionHandler- Overrides:
setOptionsin classRandomizableDensityBasedClusterer- Parameters:
options- the list of options as an array of strings- Throws:
Exception- if an option is not supported
-
numKMeansRunsTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
getNumKMeansRuns
public int getNumKMeansRuns()Returns the number of runs of k-means to perform.- Returns:
- the number of runs
-
setNumKMeansRuns
public void setNumKMeansRuns(int intValue) Set the number of runs of SimpleKMeans to perform.- Parameters:
intValue-
-
numFoldsTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setNumFolds
public void setNumFolds(int folds) Set the number of folds to use when cross-validating to find the best number of clusters.- Parameters:
folds- the number of folds to use
-
getNumFolds
public int getNumFolds()Get the number of folds to use when cross-validating to find the best number of clusters.- Returns:
- the number of folds to use
-
minLogLikelihoodImprovementCVTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setMinLogLikelihoodImprovementCV
public void setMinLogLikelihoodImprovementCV(double min) Set the minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters when cross-validating to find the best number of clusters- Parameters:
min- the minimum improvement in log likelihood
-
getMinLogLikelihoodImprovementCV
public double getMinLogLikelihoodImprovementCV()Get the minimum improvement in cross-validated log likelihood required to consider increasing the number of clusters when cross-validating to find the best number of clusters- Returns:
- the minimum improvement in log likelihood
-
minLogLikelihoodImprovementIteratingTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setMinLogLikelihoodImprovementIterating
public void setMinLogLikelihoodImprovementIterating(double min) Set the minimum improvement in log likelihood necessary to perform another iteration of the E and M steps.- Parameters:
min- the minimum improvement in log likelihood
-
getMinLogLikelihoodImprovementIterating
public double getMinLogLikelihoodImprovementIterating()Get the minimum improvement in log likelihood necessary to perform another iteration of the E and M steps.- Returns:
- the minimum improvement in log likelihood
-
numExecutionSlotsTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setNumExecutionSlots
public void setNumExecutionSlots(int slots) Set the degree of parallelism to use.- Parameters:
slots- the number of tasks to run in parallel when computing the nearest neighbors and evaluating different values of k between the lower and upper bounds
-
getNumExecutionSlots
public int getNumExecutionSlots()Get the degree of parallelism to use.- Returns:
- the number of tasks to run in parallel when computing the nearest neighbors and evaluating different values of k between the lower and upper bounds
-
displayModelInOldFormatTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setDisplayModelInOldFormat
public void setDisplayModelInOldFormat(boolean d) Set whether to display model output in the old, original format.- Parameters:
d- true if model ouput is to be shown in the old format
-
getDisplayModelInOldFormat
public boolean getDisplayModelInOldFormat()Get whether to display model output in the old, original format.- Returns:
- true if model ouput is to be shown in the old format
-
minStdDevTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setMinStdDev
public void setMinStdDev(double m) Set the minimum value for standard deviation when calculating normal density. Reducing this value can help prevent arithmetic overflow resulting from multiplying large densities (arising from small standard deviations) when there are many singleton or near singleton values.- Parameters:
m- minimum value for standard deviation
-
setMinStdDevPerAtt
public void setMinStdDevPerAtt(double[] m) -
getMinStdDev
public double getMinStdDev()Get the minimum allowable standard deviation.- Returns:
- the minumum allowable standard deviation
-
numClustersTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setNumClusters
Set the number of clusters (-1 to select by CV).- Specified by:
setNumClustersin interfaceNumberOfClustersRequestable- Parameters:
n- the number of clusters- Throws:
Exception- if n is 0
-
getNumClusters
public int getNumClusters()Get the number of clusters- Returns:
- the number of clusters.
-
setMaximumNumberOfClusters
public void setMaximumNumberOfClusters(int n) Set the maximum number of clusters to consider when cross-validating- Parameters:
n- the maximum number of clusters to consider
-
getMaximumNumberOfClusters
public int getMaximumNumberOfClusters()Get the maximum number of clusters to consider when cross-validating- Returns:
- the maximum number of clusters to consider
-
maximumNumberOfClustersTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
maxIterationsTipText
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setMaxIterations
Set the maximum number of iterations to perform- Parameters:
i- the number of iterations- Throws:
Exception- if i is less than 1
-
getMaxIterations
public int getMaxIterations()Get the maximum number of iterations- Returns:
- the number of iterations
-
debugTipText
Returns the tip text for this property- Overrides:
debugTipTextin classAbstractClusterer- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
setDebug
public void setDebug(boolean v) Set debug mode - verbose output- Overrides:
setDebugin classAbstractClusterer- Parameters:
v- true for verbose output
-
getDebug
public boolean getDebug()Get debug mode- Overrides:
getDebugin classAbstractClusterer- Returns:
- true if debug mode is set
-
getOptions
Gets the current settings of EM.- Specified by:
getOptionsin interfaceOptionHandler- Overrides:
getOptionsin classRandomizableDensityBasedClusterer- Returns:
- an array of strings suitable for passing to setOptions()
-
getClusterModelsNumericAtts
public double[][][] getClusterModelsNumericAtts()Return the normal distributions for the cluster models- Returns:
- a
double[][][]value
-
getClusterPriors
public double[] getClusterPriors()Return the priors for the clusters- Returns:
- a
double[]value
-
toString
Outputs the generated clusters into a string. -
numberOfClusters
Returns the number of clusters.- Specified by:
numberOfClustersin interfaceClusterer- Specified by:
numberOfClustersin classAbstractClusterer- Returns:
- the number of clusters generated for a training dataset.
- Throws:
Exception- if number of clusters could not be returned successfully
-
getCapabilities
Returns default capabilities of the clusterer (i.e., the ones of SimpleKMeans).- Specified by:
getCapabilitiesin interfaceCapabilitiesHandler- Specified by:
getCapabilitiesin interfaceClusterer- Overrides:
getCapabilitiesin classAbstractClusterer- Returns:
- the capabilities of this clusterer
- See Also:
-
buildClusterer
Generates a clusterer. Has to initialize all fields of the clusterer that are not being set via options.- Specified by:
buildClustererin interfaceClusterer- Specified by:
buildClustererin classAbstractClusterer- Parameters:
data- set of instances serving as training data- Throws:
Exception- if the clusterer has not been generated successfully
-
clusterPriors
public double[] clusterPriors()Returns the cluster priors.- Specified by:
clusterPriorsin interfaceDensityBasedClusterer- Specified by:
clusterPriorsin classAbstractDensityBasedClusterer- Returns:
- the cluster priors
-
logDensityPerClusterForInstance
Computes the log of the conditional density (per cluster) for a given instance.- Specified by:
logDensityPerClusterForInstancein interfaceDensityBasedClusterer- Specified by:
logDensityPerClusterForInstancein classAbstractDensityBasedClusterer- Parameters:
inst- the instance to compute the density for- Returns:
- an array containing the estimated densities
- Throws:
Exception- if the density could not be computed successfully
-
getRevision
Returns the revision string.- Specified by:
getRevisionin interfaceRevisionHandler- Overrides:
getRevisionin classAbstractClusterer- Returns:
- the revision
-
main
Main method for testing this class.- Parameters:
argv- should contain the following arguments:-t training file [-T test file] [-N number of clusters] [-S random seed]
-