
        
        
            
                
                
                    df = pd.DataFrame({"A":[3,4]})
df
                
            
               A
0  3
1  4

Let us try to modify values via chaining:


        
        
            
                
                
                    df.head(2).iloc[0,0] = 9
                
            
            1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

We encounter a SettingWithCopyWarning because the head(~) method performs slicing behind the scenes, which means that it is unclear whether a copy or a view of the DataFrame is returned.

NOTE

To briefly explain the difference between a copy and a view:

a copy of a DataFrame is a new DataFrame and modifying it would have no effect on the source DataFrame and vice versa.
a view of a DataFrame shares the same memory address as the source DataFrame, and so modifying the view will also modify the source DataFrame and vice versa.

The SettingWithCopyWarning tells us that the original DataFrame may or may not be affected. In this case, by inspecting our DataFrame df, we find that the value has been updated as well:

Now, instead of setting the value to an integer like 9, let's update the value to a floating number like 9.9:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,4]})
df.head(2).iloc[0,0] = 9.9
                
            
               A
0  3
1  4

Notice how the original df has been kept intact without any updates. The only difference between this case and the previous case is that we tried updating our column value with a floating value (9.99) instead of an integer (9).

The reason this happens is that updating the column type will cause Pandas to create a new copy of the DataFrame. In this case, column A is of type int64; updating a value in this column with another integer would not change the type of column A, and hence Pandas would directly modify the original DataFrame. In other words, the view of the DataFrame is updated. However, updating a value with a float would change the type of the column from integer to float. When this happens, Pandas creates a copy of the DataFrame and updates this copy instead of the original DataFrame.

Whether a view or a copy is updated can obviously lead to vastly different and often unexpected results, so Pandas plays it safe by always throwing a SettingWithCopyWarning if you try to update a slice of a DataFrame.

WARNING

There is a widespread misconception about views. Whether or not the original DataFrame will be mutated does not depend only on whether the method returns a copy or a view. In fact, if you check the _is_view property of the head(~) method, you will get a True:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,4]})
df.head(2)._is_view
                
            
            True

This output is somewhat misleading because this suggests that changes made to the returned DataFrame of head(2), which is a view of df, will also propagate to the original DataFrame df. However, mutations are propagated only if the column type does not change as we have demonstrated.

Possible fixes

Avoid chaining

Whenever possible, avoid chaining methods when performing an assignment. For instance, for the above case, just directly use iloc to update the values:


        
        
            
                
                
                    df.iloc[0,0] = 10
df
                
            
               A  B
0  10  6
1  4  7
2  5  8

Using loc and iloc for slicing

Instead of slicing the DataFrame with methods like head(~), you can use loc and iloc which always return a view.

To demonstrate that changes made to slices are propagated to the original DataFrame:


        
        
            
                
                
                    df.iloc[:,:].iloc[0,0] = 9.9
df
                
            
               A
0  9.90
1  4.00

Here, iloc[:,:] means we want all the rows and column, that is, we want the entire DataFrame. Next, we are using iloc again to update the value of the cell at 0,0 to 9.9. The key here is that, unlike head(~), iloc will always return a view regardless of whether the column type changes, and so the original DataFrame is always mutated.

Creating a copy

To avoid the confusion of whether changes are propagated to the original DataFrame altogether, we can always create a copy using the copy(deep=True) method:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,4]})
df_copy = df.head(2).copy(deep=True)   # Create a copy
df_copy.iloc[0,0] = 9                  # This does not affect the original df