public abstract class Learner extends Classifier
Classifier
that learns to mimic one an oracle
classifier given a feature extracting classifier and example objects.Modifier and Type | Class and Description |
---|---|
static class |
Learner.Parameters
Parameters classes are used to hold values for learning algorithm parameters,
and all learning algorithm implementations must provide a constructor that takes such an
object as input. |
Modifier and Type | Field and Description |
---|---|
protected int |
candidates
The number of candidate examples when a global object is passed here.
|
protected String |
encoding
The encoding used by this learner's feature lexicon.
|
protected Classifier |
extractor
Stores the classifiers used to produce features.
|
protected Classifier |
labeler
Stores the classifier used to produce labels.
|
protected Lexicon |
labelLexicon
Stores the label
Lexicon . |
protected URL |
lcFilePath
Caches the location of this learner's offline binary representation.
|
protected URL |
lexFilePath
Caches the location of this learner's offline lexicon.
|
protected Lexicon |
lexicon
Stores the feature
Lexicon . |
protected boolean |
lossFlag
To use a different score function based on the loss, set this flag.
|
protected FVector |
predictions
Stores the set of predictions that this learner will choose from when classifying a new
example.
|
protected boolean |
readLexiconOnDemand
Informs this learner that it can and should read its feature lexicon on demand.
|
containingPackage, name
Modifier | Constructor and Description |
---|---|
protected |
Learner()
This constructor is used by the LBJava compiler; it should never be called by a programmer.
|
protected |
Learner(String n)
Initializes the name.
|
protected |
Learner(String n,
Classifier e)
Constructor for unsupervised learning.
|
protected |
Learner(String n,
Classifier l,
Classifier e)
Constructor for supervised learning.
|
Modifier and Type | Method and Description |
---|---|
FeatureVector |
classify(FeatureVector vector)
This method makes one or more decisions about a single feature vector, returning those
decisions as
Feature s in a vector. |
FeatureVector[] |
classify(FeatureVector[] vectors)
Use this method to make a batch of classification decisions about several objects.
|
abstract FeatureVector |
classify(int[] exampleFeatures,
double[] exampleValues)
This method makes one or more decisions about a single object, returning those decisions as
Feature s in a vector. |
FeatureVector |
classify(Object example)
This method makes one or more decisions about a single object, returning those decisions as
Feature s in a vector. |
FeatureVector[] |
classify(Object[][] e)
Use this method to make a batch of classification decisions about several examples.
|
Object |
clone()
Returns a deep (enough) clone of this learner.
|
void |
countFeatures(Lexicon.CountPolicy policy)
Establishes a new feature counting policy for this learner's lexicon.
|
protected void |
createPrediction(int index)
If it hasn't been created already, this method will create the prediction feature in
predictions associated with the label feature at the given index of
labelLexicon . |
protected void |
createPrediction(Lexicon lex,
int index)
If it hasn't been created already, this method will create the prediction feature in
predictions associated with the label feature at the given index of lex
. |
Lexicon |
demandLexicon()
Forces this learner to read in its lexicon representation, but only if the lexicon currently
available in this object is empty and the learner has been scheduled to read its lexicon on
demand with
readLexiconOnDemand(URL) . |
String |
discreteValue(FeatureVector vector)
Returns the value of the discrete prediction that this learner would make, given a feature
vector.
|
String |
discreteValue(int[] f,
double[] v)
Returns the value of the discrete feature that would be returned by this classifier.
|
String |
discreteValue(Object example)
Returns the value of the discrete prediction that this learner would make, given an example.
|
void |
doneLearning()
Overridden by subclasses to perform any required post-processing computations after all
training examples have been observed through
learn(Object) and
learn(Object[]) . |
void |
doneWithRound()
Called after each round of training.
|
Learner |
emptyClone()
Returns a new, emtpy learner into which all of the parameters that control the behavior of
the algorithm have been copied.
|
Feature |
featureValue(FeatureVector vector)
Returns the classification of the given feature vector as a single feature instead of a
FeatureVector . |
Feature |
featureValue(int[] f,
double[] v)
Returns the classification of the given example as a single feature instead of a
FeatureVector . |
Feature |
featureValue(Object example)
Returns the classification of the given example object as a single feature instead of a
FeatureVector . |
void |
forget()
Reinitializes the learner to the state it started at before any learning was performed.
|
Lexicon |
getCurrentLexicon()
Returns the feature lexicon in memory, rather than reading from disk
|
Object[] |
getExampleArray(Object example)
Converts an example object into an array of arrays representing the example including its
labels.
|
Object[] |
getExampleArray(Object example,
boolean training)
Converts an example object into an array of arrays representing the example.
|
Classifier |
getExtractor()
Returns the extractor.
|
Classifier |
getLabeler()
Returns the labeler.
|
Lexicon |
getLabelLexicon()
Returns the label lexicon.
|
Lexicon |
getLexicon()
Returns the feature lexicon.
|
Lexicon |
getLexiconDiscardCounts()
Returns this learner's feature lexicon after discarding any feature counts it may have been
storing.
|
URL |
getLexiconLocation()
Returns the lexicon's location.
|
URL |
getModelLocation()
Returns the model's location.
|
Learner.Parameters |
getParameters()
Retrieves the parameters that are set in this learner.
|
int |
getPrunedLexiconSize()
Returns the size of the lexicon after any pruning that may have taken place or 0 if the
lexicon's location isn't known.
|
void |
initialize(int numExamples,
int numFeatures)
This method is sometimes called before training begins, although it is not guaranteed to be
called at all.
|
void |
learn(FeatureVector vector)
Trains the learning algorithm given a feature vector as an example.
|
void |
learn(FeatureVector[] examples)
Trains the learning algorithm given many feature vectors as examples.
|
abstract void |
learn(int[] exampleFeatures,
double[] exampleValues,
int[] exampleLabels,
double[] labelValues)
Trains the learning algorithm given an example formatted as arrays of feature indices, their
values, and the example labels.
|
void |
learn(Object example)
Trains the learning algorithm given an object as an example.
|
void |
learn(Object[] examples)
Trains the learning algorithm given many objects as examples.
|
void |
read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
Reads the binary representation of a learner with this object's run-time type, overwriting
any and all learned or manually specified parameters as well as the label lexicon but without
modifying the feature lexicon.
|
void |
read(String modelFile,
String lexFile)
Reads the learned function's binary internal represetation including both its model and
lexicons from the specified files, overwriting any and all data this object may have already
contained.
|
void |
readLabelLexicon(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
Reads the initial portion of the model file, including the containing package and name
strings, the names of the labeler and extractor, and finally the label lexicon.
|
static Learner |
readLearner(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon) from the given stream.
|
static Learner |
readLearner(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in,
boolean whole)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon), with the option of cutting off the reading process after the
label lexicon and before any learned parameters.
|
static Learner |
readLearner(String filename)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon) from the given file.
|
static Learner |
readLearner(String filename,
boolean whole)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon), with the option of cutting off the reading process after the
label lexicon and before any learned parameters.
|
static Learner |
readLearner(URL url)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon) from the given location.
|
static Learner |
readLearner(URL url,
boolean whole)
Reads the binary representation of any type of learner (including the label lexicon, but not
including the feature lexicon), with the option of cutting off the reading process after the
label lexicon and before any learned parameters.
|
void |
readLexicon(String filename)
Reads the learned function's feature lexicon from the specified file, overwriting the lexicon
present in this object, if any.
|
void |
readLexicon(URL url)
Reads the learned function's feature lexicon from the specified location, overwriting the
lexicon present in this object, if any.
|
void |
readLexiconOnDemand(String file)
Prepares this learner to read in its feature lexicon from the specified location on demand;
has no effect if this learner already has a non-empty lexicon.
|
void |
readLexiconOnDemand(URL url)
Prepares this learner to read in its feature lexicon from the specified location on demand;
has no effect if this learner already has a non-empty lexicon.
|
void |
readModel(String filename)
Reads only the learned function's model and label lexicon from the specified file in binary
form, overwriting whatever model data may have already existed in this object.
|
void |
readModel(URL url)
Reads only the learned function's model and label lexicon from the specified location in
binary form, overwriting whatever model data may have already existed in this object.
|
static Learner.Parameters |
readParameters(URL url)
Deserializes a
Learner.Parameters object out of the specified locaiton. |
double |
realValue(FeatureVector vector)
Returns the value of the real prediction that this learner would make, given a feature
vector.
|
double |
realValue(int[] f,
double[] v)
Returns the value of the real feature that would be returned by this classifier.
|
double |
realValue(Object example)
Returns the value of the real prediction that this learner would make, given an example.
|
void |
save()
Writes the binary representation of this learned function if there is a location cached in
lcFilePath , and writes the binary representation of the feature lexicon if there is
a location cached in lexFilePath . |
void |
saveLexicon()
Writes the binary representation of the feature lexicon to the location specified by
lexFilePath . |
void |
saveModel()
Writes the binary representation of this learned function to the location specified by
lcFilePath . |
ScoreSet |
scores(FeatureVector vector)
Produces a set of scores indicating the degree to which each possible discrete classification
value is associated with the given feature vector.
|
abstract ScoreSet |
scores(int[] exampleFeatures,
double[] exampleValues)
Produces a set of scores indicating the degree to which each possible discrete classification
value is associated with the given example object.
|
ScoreSet |
scores(Object example)
Produces a set of scores indicating the degree to which each possible discrete classification
value is associated with the given example object.
|
ScoreSet |
scoresAugmented(Object example,
ScoreSet resultS)
Update the score of each binary variable (label) based on the gold value of each example for
that variable.
|
void |
setCandidates(int a) |
void |
setEncoding(String e)
Sets the encoding to use in this learner's feature lexicon.
|
void |
setExtractor(Classifier e)
Sets the extractor.
|
void |
setLabeler(Classifier l)
Sets the labeler.
|
void |
setLabelLexicon(Lexicon l)
Sets the label lexicon.
|
void |
setLexicon(Lexicon l)
Sets the feature lexicon.
|
void |
setLexiconLocation(String p)
Sets the location of the lexicon as a regular file on this file system.
|
void |
setLexiconLocation(URL u)
Sets the location of the model as a
URL . |
void |
setLossFlag() |
void |
setModelLocation(String p)
Sets the location of the model as a regular file on this file system.
|
void |
setModelLocation(URL u)
Sets the location of the model as a
URL . |
void |
setParameters(Learner.Parameters p)
Sets the values of parameters that control the behavior of this learning algorithm.
|
void |
setReadLexiconOnDemand() |
void |
unclone()
Automatically generated code will override this method to set their
isClone
field to false . |
void |
unsetLossFlag() |
void |
write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)
Writes the learned function's internal representation in binary form.
|
abstract void |
write(PrintStream out)
Writes the learned function's internal representation as text.
|
void |
write(String modelFile,
String lexFile)
Writes the learned function's binary internal represetation including both its model and
lexicons to the specified files.
|
void |
writeLexicon(String filename)
Writes the learned function's feature lexicon to the specified file.
|
void |
writeModel(String filename)
Writes only the learned function's model (which includes the label lexicon) to the specified
file in binary form.
|
static void |
writeParameters(Learner.Parameters p,
String file)
Serializes a
Learner.Parameters object to the specified file. |
allowableValues, classify, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValueArray, test, toString, valueIndexOf
protected Classifier labeler
protected Classifier extractor
protected String encoding
protected FVector predictions
protected URL lcFilePath
protected URL lexFilePath
protected boolean readLexiconOnDemand
protected boolean lossFlag
protected int candidates
protected Learner()
protected Learner(String n)
n
- The name of the classifier.protected Learner(String n, Classifier e)
n
- The name of the classifier.e
- The feature extracting classifier.protected Learner(String n, Classifier l, Classifier e)
n
- The name of the classifier.l
- The labeling classifier.e
- The feature extracting classifier.public void setParameters(Learner.Parameters p)
p
- The parameters.public void setLossFlag()
public void unsetLossFlag()
public void setCandidates(int a)
public Learner.Parameters getParameters()
public void setLabeler(Classifier l)
l
- A labeling classifier.public Classifier getLabeler()
public void setExtractor(Classifier e)
e
- A feature extracting classifier.public Classifier getExtractor()
public void setLexicon(Lexicon l)
null
, the JVM's garbage collector is
invoked.l
- A feature lexicon.public Lexicon getLexicon()
public Lexicon getCurrentLexicon()
public void setLabelLexicon(Lexicon l)
l
- A feature lexicon.public Lexicon getLabelLexicon()
public void setEncoding(String e)
e
- The encoding.public void setModelLocation(String p)
p
- The file's path.public void setModelLocation(URL u)
URL
.u
- The model's location.public URL getModelLocation()
public void setLexiconLocation(String p)
p
- The file's path.public void setLexiconLocation(URL u)
URL
.u
- The model's location.public URL getLexiconLocation()
public void countFeatures(Lexicon.CountPolicy policy)
policy
- The new feature counting policy.public Lexicon getLexiconDiscardCounts()
readLexiconOnDemand(String)
or readLexiconOnDemand(URL)
has already been called, in which case the lexicon is read from disk without wasting time
loading the counts.public Learner emptyClone()
public void learn(Object example)
learn(int[],double[],int[],double[])
.example
- An example of the desired learned classifier's behavior.public void learn(FeatureVector vector)
learn(int[],double[],int[],double[])
.vector
- An example of the desired learned classifier's behavior.public abstract void learn(int[] exampleFeatures, double[] exampleValues, int[] exampleLabels, double[] labelValues)
exampleFeatures
- The example's array of feature indices.exampleValues
- The example's array of feature values.exampleLabels
- The example's label(s).labelValues
- The values of the labels.public void learn(Object[] examples)
learn(Object)
on each of the objects in the input array and finishes by
calling doneLearning()
. It should be overridden if there is a more efficient
implementation.examples
- Examples of the desired learned classifier's behavior.public void learn(FeatureVector[] examples)
learn(FeatureVector)
on each of the vectors in the input array and
finishes by calling doneLearning()
. It should be overridden if there is a more
efficient implementation.examples
- Examples of the desired learned classifier's behavior.public FeatureVector classify(Object example)
Feature
s in a vector.classify
in class Classifier
example
- The object to make decisions about.Feature
s about the input object.public FeatureVector classify(FeatureVector vector)
Feature
s in a vector.vector
- The vector to make decisions about.Feature
s about the input vector.public abstract FeatureVector classify(int[] exampleFeatures, double[] exampleValues)
Feature
s in a vector.exampleFeatures
- The example's array of feature indices.exampleValues
- The example's array of feature values.Feature
s about the input object.public FeatureVector[] classify(FeatureVector[] vectors)
classify(FeatureVector)
repeatedly) and should be overridden if there is a more
efficient implementation.vectors
- The vectors to make decisions about.public FeatureVector[] classify(Object[][] e)
classify(int[],double[])
repeatedly) and should be overridden if there is a more
efficient implementation.e
- The examples to make decisions about, represented as arrays of indices and
strengths.public Feature featureValue(Object example)
FeatureVector
.featureValue
in class Classifier
example
- The object to classify.example
as a feature.public Feature featureValue(FeatureVector vector)
FeatureVector
.vector
- The vector to classify.vector
as a feature.public Feature featureValue(int[] f, double[] v)
FeatureVector
.f
- The features array.v
- The values array.o
as a feature.public String discreteValue(Object example)
discreteValue
in class Classifier
example
- The example object.public String discreteValue(FeatureVector vector)
vector
- The example vector.public String discreteValue(int[] f, double[] v)
f
- The features array.v
- The values array.public double realValue(Object example)
realValue
in class Classifier
example
- The example object.public double realValue(FeatureVector vector)
vector
- The example vector.public double realValue(int[] f, double[] v)
f
- The features array.v
- The values array.public void doneLearning()
learn(Object)
and
learn(Object[])
. By default this method does nothing.public void initialize(int numExamples, int numFeatures)
numExamples
- The number of examples that will be observed during training.numFeatures
- The number of features that will be observed during training.public void doneWithRound()
public Object[] getExampleArray(Object example)
example
- The example object.public Object[] getExampleArray(Object example, boolean training)
training
is set to
true
. They play the same roles as the first and second arrays respectively,
except they describe the labels.example
- The example object.training
- Whether or not labels should be extracted.public void setReadLexiconOnDemand()
protected void createPrediction(int index)
predictions
associated with the label feature at the given index of
labelLexicon
. This method does not create RealFeature
s in
predictions
since their strengths cannot be modified. In association with
DiscreteFeature
s it creates a DiscretePrimitiveStringFeature
with an empty
identifier. Its value
, valueIndex
, and totalValues
fields are filled by calling the label feature's getStringValue()
, getValueIndex()
, and
totalValues()
methods respectively.index
- The index of a label feature in labelLexicon
.protected void createPrediction(Lexicon lex, int index)
predictions
associated with the label feature at the given index of lex
. This method does not create RealFeature
s in predictions
since their
strengths cannot be modified. In association with DiscreteFeature
s it creates a
DiscretePrimitiveStringFeature
with an empty identifier. Its value
,
valueIndex
, and totalValues
fields are filled by calling the label
feature's getStringValue()
, getValueIndex()
, and totalValues()
methods respectively.lex
- The label lexicon to associate prediction features with.index
- The index of a label feature in lex
.public void forget()
initialize(int,int)
to reset the number of examples and features to 0, for learners
that use this.public ScoreSet scores(Object example)
real
feature or more than one feature may implement this method by simply returning
null
.example
- The object to make decisions about.public ScoreSet scoresAugmented(Object example, ScoreSet resultS)
SparseNetworkLearner
to keep the model there is an LTU
for each label. If the gold is same as a specific label then its binary value for that label
is 1 and the score for that label will be oldScore - lossOffset
; otherwise it will be
0 and the score will be oldScore + lossOffset
.example
- The object to make decisions about.resultS
- The original scores (see scores(Object)
).public ScoreSet scores(FeatureVector vector)
real
feature or more than one feature may implement this method by simply returning
null
.vector
- The vector to make decisions about.public abstract ScoreSet scores(int[] exampleFeatures, double[] exampleValues)
real
feature or more than one feature may implement this method by simply returning
null
.exampleFeatures
- The example's array of feature indicesexampleValues
- The example's array of valuespublic abstract void write(PrintStream out)
out
- The output stream.public void unclone()
isClone
field to false
. This then allows a pure java program to read a learner's
representation into any instance of the learner's class. By default, this method does
nothing.public int getPrunedLexiconSize()
public Object clone()
lexicon
, labelLexicon
, and predictions
.
Note that this is an overriding implementation of Object
's clone()
method, and its functionality is completely separate from and unrelated to that of this
class's unclone()
method.
clone
in class Classifier
public void save()
lcFilePath
, and writes the binary representation of the feature lexicon if there is
a location cached in lexFilePath
.public void saveModel()
lcFilePath
. If lcFilePath
is not set, this method will produce an error
message and exit the program.public void saveLexicon()
lexFilePath
. If lexFilePath
is not set, this method will produce an error
message and exit the program.public void write(String modelFile, String lexFile)
lcFilePath
and
lexFilePath
.modelFile
- The name of the file in which to write the model.lexFile
- The name of the file in which to write the feature lexicon.public void writeModel(String filename)
lcFilePath
.filename
- The name of the file in which to write the model.public void writeLexicon(String filename)
lexFilePath
.filename
- The name of the file in which to write the feature lexicon.public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)
out
- The output stream.public void read(String modelFile, String lexFile)
lcFilePath
and lexFilePath
.modelFile
- The name of the file from which to read the model.lexFile
- The name of the file from which to read the feature lexicon.public void readModel(String filename)
lcFilePath
.filename
- The name of the file from which to read the model.public void readModel(URL url)
lcFilePath
.url
- The location from which to read the model.public void readLexicon(String filename)
lexFilePath
.filename
- The name of the file from which to read the feature lexicon.public void readLexicon(URL url)
lexFilePath
.url
- The location from which to read the feature lexicon.public static Learner readLearner(String filename)
-1
appears instead, this method returns null
.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
filename
- The name of the file from which to read the learner.public static Learner readLearner(String filename, boolean whole)
whole
is
false
, the reading process is cut off in this way.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
filename
- The name of the file from which to read the learner.whole
- Whether or not to read the whole model.public static Learner readLearner(URL url)
-1
appears instead, this method returns null
.
Finally, the location is cached in lcFilePath
.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
url
- The location from which to read the learner.public static Learner readLearner(URL url, boolean whole)
whole
is
false
, the reading process is cut off in this way. Finally, the location is
cached in lcFilePath
.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
url
- The location from which to read the learner.whole
- Whether or not to read the whole model.public static Learner readLearner(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
-1
appears instead, this method returns null
.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
in
- The input stream.public static Learner readLearner(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in, boolean whole)
whole
is
false
, the reading process is cut off in this way.
This method is appropriate for reading learners as written by
write(ExceptionlessOutputStream)
.
in
- The input stream.whole
- Whether or not to read the whole model.public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
in
- The input stream.public void readLabelLexicon(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)
in
- The input stream.public void readLexiconOnDemand(String file)
file
- The file from which to read the feature lexicon.public void readLexiconOnDemand(URL url)
url
- The location from which to read the feature lexicon.public Lexicon demandLexicon()
readLexiconOnDemand(URL)
.lexicon
.readLexiconOnDemand
public static void writeParameters(Learner.Parameters p, String file)
Learner.Parameters
object to the specified file.p
- The parameters to serialize.file
- The file in which to serialize them.public static Learner.Parameters readParameters(URL url)
Learner.Parameters
object out of the specified locaiton.url
- The location from which to read the object.Copyright © 2016. All rights reserved.