https://ask.ghassem.com/tag/sql Powered by Question2Answer https://ask.ghassem.com/979/terminology-clarification-in-spark <p>I have a hard time distinguishing&nbsp;terminologies&nbsp;of SparkSQL. While SparkSQL are quite flexible in terms of abstraction layers, its really difficult for beginner to navigate around those options.</p> <p>1. When we say " using SparkSQL to perform .....", does it mean that we can use any API/abstraction layers such as Scala, Python, HiveQL to query? As long as the core dataframe is in spark, we should be fine?</p> <p>2. Can we manipulate data in both PySpark and Scala sequentially?</p> <p>For example, may I clean up the data in Scala, then perform follow up manipulation in PySpark, then go back to Scala?</p> <p>3. As demonstrated in the tutorial, we can query&nbsp;with SQL command by using the api spark.sql("My SQL command"). does it count as SQL or SPARK?</p> <p>&nbsp;</p> <p><img alt="Image result for spark sql" src="http://cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/f4a5b21d-66fa-4885-92bf-c4e81c06d916/Image/753ace3c801b53535077d9474ecc5f1e/odi_spark_sql_databricks.jpg"></p> Big Data Tools https://ask.ghassem.com/979/terminology-clarification-in-spark Sat, 06 Feb 2021 18:03:32 +0000 https://ask.ghassem.com/7/what-are-the-best-resources-for-studying-sql I am wondering if anyone can suggest the best resources for studying SQL? SQL https://ask.ghassem.com/7/what-are-the-best-resources-for-studying-sql Sun, 26 Aug 2018 07:58:51 +0000