
        
        
            
                
                
                    df = spark.createDataFrame([[20,10], [30,50], [40,90]], ["A", "B"])
df.show()
                
            
            +---+---+
|  A|  B|
+---+---+
| 20| 10|
| 30| 50|
| 40| 90|
+---+---+

Getting the least value for every row of specified columns in PySpark

To get the least value for every row of columns A and B:


        
        
            
                
                
                    import pyspark.sql.functions as F
df.select(F.least("A","B")).show()
                
            
            +-----------+
|least(A, B)|
+-----------+
|         10|
|         30|
|         40|
+-----------+

We can also pass Column objects instead of column labels:


        
        
            
                
                
                    df.select(F.least(df.A,df.B)).show()
                
            
            +-----------+
|least(A, B)|
+-----------+
|         10|
|         30|
|         40|
+-----------+

Note that we can append the Column returned by least(~) by using withColumn(~):


        
        
            
                
                
                    df.withColumn("smallest", F.least("A","B")).show()
                
            
            +---+---+--------+
|  A|  B|smallest|
+---+---+--------+
| 20| 10|      10|
| 30| 50|      30|
| 40| 90|      40|
+---+---+--------+

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

Official PySpark Documentation

https://spark.apache.org/docs/3.2.0/api/python/reference/api/pyspark.sql.functions.least.html

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!