Package org.apache.spark.ml.feature

Class VectorAssembler

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Transformer
org.apache.spark.ml.feature.VectorAssembler
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, Params, HasHandleInvalid, HasInputCols, HasOutputCol, DefaultParamsWritable, Identifiable, MLWritable
public class VectorAssembler extends Transformer implements HasInputCols, HasOutputCol, HasHandleInvalid, DefaultParamsWritable
A feature transformer that merges multiple columns into a vector column.

This requires one pass over the entire dataset. In case we need to infer column lengths from the data we require an additional call to the 'first' Dataset method, see 'handleInvalid' parameter.

See Also:

  • Constructor Details

    • VectorAssembler

      public VectorAssembler(String uid)

    • VectorAssembler

      public VectorAssembler()

  • Method Details

    • load

      public static VectorAssembler load(String path)

    • read

      public static MLReader<T> read()

    • outputCol

      public final Param<String> outputCol()
      Description copied from interface: HasOutputCol
      Param for output column name.
      Specified by:
      outputCol in interface HasOutputCol
      Returns:
      (undocumented)

    • inputCols

      public final StringArrayParam inputCols()
      Description copied from interface: HasInputCols
      Param for input column names.
      Specified by:
      inputCols in interface HasInputCols
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setInputCols

      public VectorAssembler setInputCols(String[] value)

    • setOutputCol

      public VectorAssembler setOutputCol(String value)

    • setHandleInvalid

      public VectorAssembler setHandleInvalid(String value)

    • handleInvalid

      public Param<String> handleInvalid()
      Param for how to handle invalid data (NULL values). Options are 'skip' (filter out rows with invalid data), 'error' (throw an error), or 'keep' (return relevant number of NaN in the output). Column lengths are taken from the size of ML Attribute Group, which can be set using VectorSizeHint in a pipeline before VectorAssembler. Column lengths can also be inferred from first rows of the data since it is safe to do so but only in case of 'error' or 'skip'. Default: "error"
      Specified by:
      handleInvalid in interface HasHandleInvalid
      Returns:
      (undocumented)

    • transform

      public Dataset<Row> transform(Dataset<?> dataset)
      Description copied from class: Transformer
      Transforms the input dataset.
      Specified by:
      transform in class Transformer
      Parameters:
      dataset - (undocumented)
      Returns:
      (undocumented)

    • transformSchema

      public StructType transformSchema(StructType schema)
      Description copied from class: PipelineStage
      Check transform validity and derive the output schema from the input schema.

      We check validity for interactions between parameters during transformSchema and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by Param.validate().

      Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.

      Specified by:
      transformSchema in class PipelineStage
      Parameters:
      schema - (undocumented)
      Returns:
      (undocumented)

    • copy

      public VectorAssembler copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Transformer
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)

    • toString

      public String toString()
      Specified by:
      toString in interface Identifiable
      Overrides:
      toString in class Object