Package dev.vortex.spark.read

Class VortexPartitionReader

java.lang.Object
dev.vortex.spark.read.VortexPartitionReader
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch>
public final class VortexPartitionReader extends Object implements org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch>
Columnar reader over one Spark PartitionedFile.

  • Constructor Summary

    Constructors
    Constructor
    Description
    VortexPartitionReader(org.apache.spark.sql.execution.datasources.PartitionedFile file, org.apache.spark.sql.types.StructType dataSchema, org.apache.spark.sql.types.StructType readDataSchema, org.apache.spark.sql.types.StructType readPartitionSchema, VortexIo io, VortexOptions formatOptions, org.apache.spark.sql.sources.Filter[] pushedFilters, boolean caseSensitive)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    org.apache.spark.sql.vectorized.ColumnarBatch
    get()
     
    boolean
    next()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.read.PartitionReader

    currentMetricsValues, initMetricsValues

  • Constructor Details

    • VortexPartitionReader

      public VortexPartitionReader(org.apache.spark.sql.execution.datasources.PartitionedFile file, org.apache.spark.sql.types.StructType dataSchema, org.apache.spark.sql.types.StructType readDataSchema, org.apache.spark.sql.types.StructType readPartitionSchema, VortexIo io, VortexOptions formatOptions, org.apache.spark.sql.sources.Filter[] pushedFilters, boolean caseSensitive)

  • Method Details

    • next

      public boolean next()
      Specified by:
      next in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch>

    • get

      public org.apache.spark.sql.vectorized.ColumnarBatch get()
      Specified by:
      get in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch>

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable