Question: I See Spark 3.x is available, is Cloudera CDH 6.x support this version?
Answer: No, Spark 3.x is not supported on CDH 6.3 as of now. The latest Spark version Spark 2.4 is supported as part of Cloudera CDH 6.3 release. Even you should know that from Spark 2.4 not all the things are supported like Continuous Stream is not supported on CDH 6.3, Standalone Spark Cluster Manager is not supported. However, new things like below are supported.
- DataFrame/DataSet
- Catalyst Optimizer
- Structured Streaming
Even as per the documentation CDH 6 does not support the multiple version of the Spark, and as soon as you upgrade to the CDH6 it disables the CDS 2 parcel if they exists, and all Spark Services will use the built-in Spark Version which is Spark 2.4 as of now.