from pyspark.sql.types import *
schema = StructType([StructField("col1",StringType(),True),\
StructField("col2",IntegerType(),True),\
StructField("col3",DoubleType(),True),\
StructField("col4",FloatType(),True)])
rowdata = []
rowdata.append(("col1_value",222,1.23,3.44555))
...
createDf = spark.createDataFrame(rowdata,schema)
'spark' 카테고리의 다른 글
[spark dataframe] extract date value using pyspark udf lambda (0) | 2021.12.27 |
---|---|
[spark] dataframe get row value 데이터 전처리 (0) | 2020.06.16 |