Package dev.vortex.spark.write

Class VortexOutputWriterFactory

java.lang.Object

org.apache.spark.sql.execution.datasources.OutputWriterFactory

dev.vortex.spark.write.VortexOutputWriterFactory

All Implemented Interfaces:

Serializable

public final class VortexOutputWriterFactory
extends org.apache.spark.sql.execution.datasources.OutputWriterFactory

Creates Vortex output writers at paths assigned by Spark's file commit protocol.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
VortexOutputWriterFactory(org.apache.spark.sql.types.StructType schema, VortexOptions options)

Method Summary

Modifier and Type	Method	Description
String	getFileExtension(org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.spark.sql.execution.datasources.OutputWriter	newInstance(String path, org.apache.spark.sql.types.StructType dataSchema, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VortexOutputWriterFactory

public VortexOutputWriterFactory(org.apache.spark.sql.types.StructType schema,
 VortexOptions options)

Method Details

getFileExtension

public String getFileExtension(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Specified by:

getFileExtension in class org.apache.spark.sql.execution.datasources.OutputWriterFactory

newInstance

public org.apache.spark.sql.execution.datasources.OutputWriter newInstance(String path,
 org.apache.spark.sql.types.StructType dataSchema,
 org.apache.hadoop.mapreduce.TaskAttemptContext context)

Specified by:

newInstance in class org.apache.spark.sql.execution.datasources.OutputWriterFactory