Pandas Groupby Percentile - Python

When working with large datasets, understanding the distribution of the data is crucial. One way to track this distribution is by calculating percentiles. In this tutorial, we'll explore how to use the groupby function in Pandas along with the quantile method to calculate percentiles for each group.

Grouping Data

First, let's create a sample dataset to work with:

import pandas as pd
import numpy as np

np.random.seed(0)

df = pd.DataFrame({
    'group': ['A', 'A', 'B', 'B', 'C', 'C'],
    'value': np.random.randn(6)
})

print(df)

# Output:
#   group     value
# 0     A  1.764052
# 1     A  0.400157
# 2     B  0.978738
# 3     B  2.240893
# 4     C  1.867558
# 5     C -0.977278

Our dataset consists of 6 rows of random data with a "group" column and a "value" column.

Next, we'll group the data by the "group" column:

grouped = df.groupby('group')

Calculating Percentiles

Now that we've grouped the data, we can calculate percentiles for each group using the quantile method. Let's calculate the 25th and 75th percentiles for each group:

percentiles = grouped['value'].quantile([0.25, 0.75])
print(percentiles)

# Output:
#           0.25      0.75
# group                  
# A      0.582079  1.082105
# B      0.909263  1.609315
# C      0.445640  1.656099

Here, we've passed 0.25 and 0.75 as arguments to the quantile method to calculate the 25th and 75th percentiles. The resulting dataframe shows the percentiles for each group in the "group" column.

Conclusion

Calculating percentiles for grouped data is easy with Pandas. By using the groupby function along with the quantile method, we can quickly calculate percentiles for each group. This is useful when analyzing large datasets to understand the distribution of the data.