To do it in a spark-shell (Scala), you can use spark.time()
.
See another response by me: https://stackoverflow.com/a/50289329/3397114
df = sqlContext.sql(query)
spark.time(df.show())
The output would be:
+----+----+
|col1|col2|
+----+----+
|val1|val2|
+----+----+
Time taken: xxx ms
Related: On Measuring Apache Spark Workload Metrics for Performance Troubleshooting.
Related Contents:
- How to select the first row of each group?
- Querying Spark SQL DataFrame with complex types
- Concatenate columns in Apache Spark DataFrame
- Spark Window Functions – rangeBetween dates
- How to store only time; not date and time?
- How to convert an integer (time) to HH:MM:SS::00 in SQL Server 2008?
- Does SparkSQL support subquery?
- Including null values in an Apache Spark Join
- Spark SQL window function with complex condition
- Spark sql queries vs dataframe functions
- Difference between datetime and timestamp in sqlserver? [duplicate]
- Explode (transpose?) multiple columns in Spark SQL table
- SPARK SQL – case when then
- How to get other columns when using Spark DataFrame groupby?
- How do I get a SQL row_number equivalent for a Spark RDD?
- get TopN of all groups after group by using Spark DataFrame
- Why using a UDF in a SQL query leads to cartesian product?
- How do I obtain a Query Execution Plan in SQL Server?
- How do I perform an IF…THEN in an SQL SELECT?
- Auto Increment after delete in MySQL
- Comparing date ranges
- In SQL, how can you “group by” in ranges?
- How do I spool to a CSV formatted file using SQLPLUS?
- MIN/MAX vs ORDER BY and LIMIT
- How to Create a real one-to-one relationship in SQL Server
- Escaping keyword-like column names in Postgres
- How can I import a JSON file into PostgreSQL?
- Is it possible to output a SELECT statement from a PL/SQL block?
- SQLite – How do you join tables from different databases?
- Does Foreign Key improve query performance?
- Storing money in a decimal column – what precision and scale? [closed]
- What does a question mark represent in SQL queries?
- Find the smallest unused number in SQL Server
- Fastest check if row exists in PostgreSQL
- How do I use ROW_NUMBER()?
- How to query a CLOB column in Oracle
- How do I remove redundant namespace in nested query when using FOR XML PATH
- unresolved reference to object [INFORMATION_SCHEMA].[TABLES]
- Get top row(s) with highest value, with ties
- How to round an average to 2 decimal places in PostgreSQL?
- How can I alter this computed column in SQL Server 2008?
- Is there a SQL implementation of PBKDF2?
- MySQL #1140 – Mixing of GROUP columns
- Adding a new SQL column with a default value
- SQL performance: WHERE vs WHERE(ROW_NUMBER)
- Generate nested nth level JSON SQL Server using recursive CTE
- Update query using Subquery in Sql Server
- “column not allowed here” error in INSERT statement
- sql query joins multiple tables – too slow (8 tables)
- Inserting into Oracle and retrieving the generated sequence ID