PySpark is a wrapper, so Scala is unnecessary and boggy. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tomrod 8 months ago \| parent \| context \| favorite \| on: A lost decade chasing distributed architectures fo... PySpark is a wrapper, so Scala is unnecessary and boggy.

spark1377485 8 months ago [–]

PySpark is great, except for UDF performance. This gap means that Scala is helpful for some Spark edge cases like column-level encryption/decryption with UDF

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact