Package weka.core.converters
Class TextDirectoryLoader
java.lang.Object
weka.core.converters.AbstractLoader
weka.core.converters.TextDirectoryLoader
- All Implemented Interfaces:
Serializable,CommandlineRunnable,BatchConverter,IncrementalConverter,Loader,OptionHandler,RevisionHandler
public class TextDirectoryLoader
extends AbstractLoader
implements BatchConverter, IncrementalConverter, OptionHandler, CommandlineRunnable
Loads all text files in a directory and uses the
subdirectory names as class labels. The content of the text files will be
stored in a String attribute, the filename can be stored as well.
Valid options are:
Wiki article
-D Enables debug output. (default: off)
-F Stores the filename in an additional attribute. (default: off)
-dir <directory> The directory to work on. (default: current directory)
-charset <charset name> The character set to use, e.g UTF-8. (default: use the default character set)
-R Retain all string attribute values when reading incrementally.Based on code from the TextDirectoryToArff tool. See the
- Version:
- $Revision: 15255 $
- Author:
- Ashraf M. Kibriya (amk14 at cs.waikato.ac.nz), Richard Kirkby (rkirkby at cs.waikato.ac.nz), fracpete (fracpete at waikato dot ac dot nz)
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from interface weka.core.converters.Loader
Loader.StructureNotReadyException -
Field Summary
Fields inherited from interface weka.core.converters.Loader
BATCH, INCREMENTAL, NONE -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionthe tip text for this propertythe tip text for this propertyGet the character set to use when reading text files.Return the full data set.booleangetDebug()Gets whether additional debug information is printed.get the Dir specified as the sourceReturns a description of the file type, actually it's directories.getNextInstance(Instances structure) Process input directories/files incrementally.String[]Gets the settingbooleanGets whether the filename will be stored as an extra attribute.Returns the revision string.Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.Returns a string describing this loaderLists the available optionsstatic voidMain method.the tip text for this propertyvoidPerform any teardown stuff that might need to happen after execution.voidPerform any setup stuff that might need to happen before commandline execution.voidreset()Resets the loader ready to read a new data setvoidExecute the supplied object.voidsetCharSet(String charSet) Set the character set to use when reading text files (an empty string indicates that the default character set will be used).voidsetDebug(boolean value) Sets whether to print some debug information.voidsetDirectory(File dir) sets the source directoryvoidsetOptions(String[] options) Parses a given list of options.voidsetOutputFilename(boolean value) Sets whether the filename will be stored as an extra attribute.voidResets the Loader object and sets the source of the data set to be the supplied File object.Methods inherited from class weka.core.converters.AbstractLoader
setRetrieval, setSource
-
Constructor Details
-
TextDirectoryLoader
public TextDirectoryLoader()default constructor
-
-
Method Details
-
globalInfo
Returns a string describing this loader- Returns:
- a description of the evaluator suitable for displaying in the explorer/experimenter gui
-
listOptions
Lists the available options- Specified by:
listOptionsin interfaceOptionHandler- Returns:
- an enumeration of the available options
-
setOptions
Parses a given list of options. Valid options are:-D Enables debug output. (default: off)
-F Stores the filename in an additional attribute. (default: off)
-dir <directory> The directory to work on. (default: current directory)
-charset <charset name> The character set to use, e.g UTF-8. (default: use the default character set)
- Specified by:
setOptionsin interfaceOptionHandler- Parameters:
options- the options- Throws:
Exception- if options cannot be set
-
getOptions
Gets the setting- Specified by:
getOptionsin interfaceOptionHandler- Returns:
- the current setting
-
charSetTipText
the tip text for this property- Returns:
- the tip text
-
setCharSet
Set the character set to use when reading text files (an empty string indicates that the default character set will be used).- Parameters:
charSet- the character set to use.
-
getCharSet
Get the character set to use when reading text files. An empty string indicates that the default character set will be used.- Returns:
- the character set name to use (or empty string to indicate that the default character set will be used).
-
setDebug
public void setDebug(boolean value) Sets whether to print some debug information.- Parameters:
value- if true additional debug information will be printed.
-
getDebug
public boolean getDebug()Gets whether additional debug information is printed.- Returns:
- true if additional debug information is printed
-
debugTipText
the tip text for this property- Returns:
- the tip text
-
setOutputFilename
public void setOutputFilename(boolean value) Sets whether the filename will be stored as an extra attribute.- Parameters:
value- if true the filename will be stored in an extra attribute
-
getOutputFilename
public boolean getOutputFilename()Gets whether the filename will be stored as an extra attribute.- Returns:
- true if the filename is stored in an extra attribute
-
outputFilenameTipText
the tip text for this property- Returns:
- the tip text
-
getFileDescription
Returns a description of the file type, actually it's directories.- Returns:
- a short file description
-
getDirectory
get the Dir specified as the source- Returns:
- the source directory
-
setDirectory
sets the source directory- Parameters:
dir- the source directory- Throws:
IOException- if an error occurs
-
reset
public void reset()Resets the loader ready to read a new data set- Specified by:
resetin interfaceLoader- Overrides:
resetin classAbstractLoader
-
setSource
Resets the Loader object and sets the source of the data set to be the supplied File object.- Specified by:
setSourcein interfaceLoader- Overrides:
setSourcein classAbstractLoader- Parameters:
dir- the source directory.- Throws:
IOException- if an error occurs
-
getStructure
Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.- Specified by:
getStructurein interfaceLoader- Specified by:
getStructurein classAbstractLoader- Returns:
- the structure of the data set as an empty set of Instances
- Throws:
IOException- if an error occurs
-
getDataSet
Return the full data set. If the structure hasn't yet been determined by a call to getStructure then method should do so before processing the rest of the data set.- Specified by:
getDataSetin interfaceLoader- Specified by:
getDataSetin classAbstractLoader- Returns:
- the structure of the data set as an empty set of Instances
- Throws:
IOException- if there is no source or parsing fails
-
getNextInstance
Process input directories/files incrementally.- Specified by:
getNextInstancein interfaceLoader- Specified by:
getNextInstancein classAbstractLoader- Parameters:
structure- ignored- Returns:
- never returns without throwing an exception
- Throws:
IOException- if a problem occurs
-
getRevision
Returns the revision string.- Specified by:
getRevisionin interfaceRevisionHandler- Returns:
- the revision
-
main
Main method.- Parameters:
args- should contain the name of an input file.
-
preExecution
Perform any setup stuff that might need to happen before commandline execution. Subclasses should override if they need to do something here- Specified by:
preExecutionin interfaceCommandlineRunnable- Throws:
Exception- if a problem occurs during setup
-
postExecution
Perform any teardown stuff that might need to happen after execution. Subclasses should override if they need to do something here- Specified by:
postExecutionin interfaceCommandlineRunnable- Throws:
Exception- if a problem occurs during teardown
-
run
Description copied from interface:CommandlineRunnableExecute the supplied object.- Specified by:
runin interfaceCommandlineRunnable- Parameters:
toRun- the object to executeargs- any options to pass to the object- Throws:
IllegalArgumentException
-