pyspark.sql.functions.input_file_block_start

pyspark.sql.functions.input_file_block_start()

Returns the start offset of the block being read, or -1 if not available.

New in version 3.5.0.

See also

pyspark.sql.functions.input_file_name()

pyspark.sql.functions.input_file_block_length()

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.read.text("python/test_support/sql/ages_newlines.csv", lineSep=",")
>>> df.select(sf.input_file_block_start()).show()
+------------------------+
|input_file_block_start()|
+------------------------+
|                       0|
|                       0|
|                       0|
|                       0|
|                       0|
|                       0|
|                       0|
|                       0|
+------------------------+