python dataframe conditional assignment

Data to Fish

5 ways to apply an IF condition in Pandas DataFrame

In this guide, you’ll see 5 different ways to apply an IF condition in Pandas DataFrame.

Specifically, you’ll see how to apply an IF condition for:

• Set of numbers
• Set of numbers and lambda
• Strings and lambda
• OR condition

Applying an IF condition in Pandas DataFrame

Let’s now review the following 5 cases:

(1) IF condition – Set of numbers

Suppose that you created a DataFrame in Python that has 10 numbers (from 1 to 10). You then want to apply the following IF conditions:

• If the number is equal or lower than 4, then assign the value of ‘ Yes ‘
• Otherwise, if the number is greater than 4, then assign the value of ‘ No ‘

This is the general structure that you may use to create the IF condition:

For our example, the Python code would look like this:

Here is the result that you’ll get in Python:

(2) IF condition – set of numbers and  lambda

You’ll now see how to get the same results as in case 1 by using lambda, where the conditions are:

Here is the generic structure that you may apply in Python:

For our example:

This is the result that you’ll get, which matches with case 1:

(3) IF condition – strings

Now, let’s create a DataFrame that contains only strings/text with 4  names : Jon, Bill, Maria and Emma.

The conditions are:

• If the name is equal to ‘Bill,’ then assign the value of ‘ Match ‘
• Otherwise, if the name is not   ‘Bill,’ then assign the value of ‘ Mismatch ‘

Once you run the above Python code, you’ll see:

(4) IF condition – strings and lambda 

You’ll get the same results as in case 3 by using lambda:

And here is the output from Python:

(5) IF condition with OR

Now let’s apply these conditions:

• If the name is ‘Bill’  or ‘Emma,’ then assign the value of ‘ Match ‘
• Otherwise, if the name is neither ‘Bill’ nor ‘Emma,’ then assign the value of ‘ Mismatch ‘

Run the Python code, and you’ll get the following result:

Applying an IF condition under an existing DataFrame column

So far you have seen how to apply an IF condition by creating a new column.

Alternatively, you may store the results under an existing DataFrame column.

For example, let’s say that you created a DataFrame that has 12 numbers, where the last two numbers are zeros :

‘set_of_numbers’: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0 , 0 ]

You may then apply the following IF conditions, and then store the results under the existing ‘ set_of_numbers ‘ column:

• If the number is equal to 0 , then change the value to 999
• If the number is equal to 5 , then change the value to 555

Here are the before and after results, where the ‘5’ became ‘555’ and the 0’s became ‘999’ under the existing ‘set_of_numbers’ column:

On another instance, you may have a DataFrame that contains NaN values . You can then apply an IF condition to replace those values with zeros , as in the example below:

Before you’ll see the NaN values, and after you’ll see the zero values:

You just saw how to apply an IF condition in Pandas DataFrame . There are indeed multiple ways to apply such a condition in Python. You can achieve the same results by using either lambda, or just by sticking with Pandas.

At the end, it boils down to working with the method that is best suited to your needs.

Finally, you may want to check the following external source for additional information about Pandas DataFrame .

• Learn Python
• Python Lists
• Python Dictionaries
• Python Strings
• Python Functions
• Learn Pandas & NumPy
• Pandas Tutorials
• Numpy Tutorials
• Learn Data Visualization
• Python Seaborn
• Python Matplotlib

Set Pandas Conditional Column Based on Values of Another Column

• August 9, 2021 February 22, 2022

There are many times when you may need to set a Pandas column value based on the condition of another column. In this post, you’ll learn all the different ways in which you can create Pandas conditional columns.

Table of Contents

Video Tutorial

If you prefer to follow along with a video tutorial, check out my video below:

Loading a Sample Dataframe

Let’s begin by loading a sample Pandas dataframe that we can use throughout this tutorial.

We’ll begin by import pandas and loading a dataframe using the .from_dict() method:

This returns the following dataframe:

Using Pandas loc to Set Pandas Conditional Column

Pandas loc is incredibly powerful! If you need a refresher on loc (or iloc), check out my tutorial here . Pandas’ loc creates a boolean mask, based on a condition. Sometimes, that condition can just be selecting rows and columns, but it can also be used to filter dataframes. These filtered dataframes can then have values applied to them.

Let’s explore the syntax a little bit:

With the syntax above, we filter the dataframe using .loc and then assign a value to any row in the column (or columns) where the condition is met.

Let’s try this out by assigning the string ‘Under 30’ to anyone with an age less than 30, and ‘Over 30’ to anyone 30 or older.

Let's take a look at what we did here:

• We assigned the string 'Over 30' to every record in the dataframe. To learn more about this, check out my post here or creating new columns.
• We then use .loc to create a boolean mask on the Age column to filter down to rows where the age is less than 30. When this condition is met, the Age Category column is assigned the new value 'Under 30'

But what happens when you have multiple conditions? You could, of course, use .loc multiple times, but this is difficult to read and fairly unpleasant to write. Let's see how we can accomplish this using numpy's .select() method.

Using Numpy Select to Set Values using Multiple Conditions

Similar to the method above to use .loc to create a conditional column in Pandas, we can use the numpy .select() method.

Let's begin by importing numpy and we'll give it the conventional alias np :

Now, say we wanted to apply a number of different age groups, as below:

• <20 years old,
• 20-39 years old,
• 40-59 years old,
• 60+ years old

In order to do this, we'll create a list of conditions and corresponding values to fill:

Running this returns the following dataframe:

Let's break down what happens here:

• We first define a list of conditions in which the criteria are specified. Recall that lists are ordered meaning that they should be in the order in which you would like the corresponding values to appear.
• We then define a list of values to use , which corresponds to the values you'd like applied in your new column.

Something to consider here is that this can be a bit counterintuitive to write. You can similarly define a function to apply different values. We'll cover this off in the section of using the Pandas .apply() method below .

One of the key benefits is that using numpy as is very fast, especially when compared to using the .apply() method.

Using Pandas Map to Set Values in Another Column

The Pandas .map() method is very helpful when you're applying labels to another column. In order to use this method, you define a dictionary to apply to the column.

For our sample dataframe, let's imagine that we have offices in America, Canada, and France. We want to map the cities to their corresponding countries and apply and "Other" value for any other city.

When we print this out, we get the following dataframe returned:

What we can see here, is that there is a NaN value associated with any City that doesn't have a corresponding country. If we want to apply "Other" to any missing values, we can chain the .fillna() method:

Using Pandas Apply to Apply a function to a column

Finally, you can apply built-in or custom functions to a dataframe using the Pandas .apply() method.

Let's take a look at both applying built-in functions such as len() and even applying custom functions.

Applying Python Built-in Functions to a Column

We can easily apply a built-in function using the .apply() method. Let's see how we can use the len() function to count how long a string of a given column.

Take note of a few things here:

• We apply the .apply() method to a particular column,
• We omit the parentheses "()"

Using Third-Party Packages in Pandas Apply

Similarly, you can use functions from using packages. Let's use numpy to apply the .sqrt() method to find the scare root of a person's age.

Using Custom Functions with Pandas Apply

Something that makes the .apply() method extremely powerful is the ability to define and apply your own functions.

Let's revisit how we could use an if-else statement to create age categories as in our earlier example:

In this post, you learned a number of ways in which you can apply values to a dataframe column to create a Pandas conditional column, including using .loc , .np.select() , Pandas .map() and Pandas .apply() . Each of these methods has a different use case that we explored throughout this post.

Learn more about Pandas methods covered here by checking out their official documentation:

• Pandas Apply
• Numpy Select

Nik Piepenbreier

Nik is the author of datagy.io and has over a decade of experience working with data analytics, data science, and Python. He specializes in teaching developers how to use Python for data science using hands-on tutorials. View Author posts

2 thoughts on “Set Pandas Conditional Column Based on Values of Another Column”

Thank you so much! Brilliantly explained!!!

Thanks Aisha!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Free Python 3 Tutorial
• Control Flow
• Exception Handling
• Python Programs
• Python Projects
• Python Interview Questions
• Python Database
• Data Science With Python
• Machine Learning with Python
• Split Pandas Dataframe by column value
• Add multiple columns to dataframe in Pandas
• Rename specific column(s) in Pandas
• Applying Lambda functions to Pandas Dataframe
• How to Create a Pivot table with multiple indexes from an excel sheet using Pandas in Python?
• Convert a series of date strings to a time series in Pandas Dataframe
• How to Count the NaN Occurrences in a Column in Pandas Dataframe?
• Pandas remove rows with special characters
• How to Count Distinct Values of a Pandas Dataframe Column?
• Get Seconds from timestamp in Python-Pandas
• Display the Pandas DataFrame in table style and border around the table and not around the rows
• How to Iterate over Dataframe Groups in Python-Pandas?
• How to get the powers of an array values element-wise in Python-Pandas?
• How to Select single column of a Pandas Dataframe?
• How to widen output display to see more columns in Pandas dataframe?
• Convert Floats to Integers in a Pandas DataFrame
• Check if a column starts with given string in Pandas DataFrame?
• How to find the Cross-section of Pandas Data frame?
• How to Filter DataFrame Rows Based on the Date in Pandas?

Ways to apply an if condition in Pandas DataFrame

Generally on a Pandas DataFrame the if condition can be applied either column-wise, row-wise, or on an individual cell basis. The further document illustrates each of these with examples.

First of all we shall create the following DataFrame : 

Output : 

Example 1 : if condition on column values (tuples) : The if condition can be applied on column values like when someone asks for all the items with the MRP <=2000 and Discount >0 the following code does that. Similarly, any number of conditions can be applied on any number of attributes of the DataFrame. 

Example 2 : if condition on row values (tuples) : This can be taken as a special case for the condition on column values. If a tuple is given (Sofa, 5000, 20) and finding it in the DataFrame can be done like :

Example 3 : Using Lambda function : Lambda function takes an input and returns a result based on a certain condition. It can be used to apply a certain function on each of the elements of a column in Pandas DataFrame. The below example uses the Lambda function to set an upper limit of 20 on the discount value i.e. if the value of discount > 20 in any cell it sets it to 20.

Example 4 : Using iloc() or loc() function : Both iloc() and loc() function are used to extract the sub DataFrame from a DataFrame. The sub DataFrame can be anything spanning from a single cell to the whole table. iloc() is generally used when we know the index range for the row and column whereas loc() is used on a label search.

The below example shows the use of both of the functions for imparting conditions on the Dataframe. Here a cell with index [2, 1] is taken which is the Badminton product’s MRP. 

Please Login to comment...

Similar reads.

• Python pandas-dataFrame
• Python Pandas-exercise
• Python-pandas
• CBSE Exam Format Changed for Class 11-12: Focus On Concept Application Questions
• 10 Best Waze Alternatives in 2024 (Free)
• 10 Best Squarespace Alternatives in 2024 (Free)
• Top 10 Owler Alternatives & Competitors in 2024
• 30 OOPs Interview Questions and Answers (2024)

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

How to Apply the If-Else Condition in a Pandas DataFrame

• Python Pandas Howtos
• How to Apply the If-Else Condition in a …

Use DataFrame.loc[] to Apply the if-else Condition in a Pandas DataFrame in Python

Use dataframe.apply() to apply the if-else condition in a pandas dataframe in python, use numpy.select() to apply the if-else condition in a pandas dataframe in python, use lambda with apply() to apply the if-else condition in a pandas dataframe in python.

Pandas is an open-source data analysis library in Python. It provides many built-in methods to perform operations on numerical data.

In some cases, we want to apply the if-else conditions on a Pandas dataframe to filter the records or perform computations according to some conditions. Python provides many ways to use if-else on a Pandas dataframe.

loc[] is a property of the Pandas data frame used to select or filter a group of rows or columns. In the following example, we will employ this property to filter the records that meet a given condition.

Here, we have a Pandas data frame consisting of the students’ data. Using loc[] , we can only apply a single condition at a time.

We will filter those students having marks greater than or equal to 60 in the first condition and assign their result as Pass in the new column Result . Similarly, we will set Fail for the rest of the student’s results in another condition.

Example Code:

The apply() method uses the data frame’s axis (row or column) to apply a function. We can make our defined function that consists of if-else conditions and apply it to the Pandas dataframe.

Here, we have defined a function assign_Result() and applied it to the Marks column. The function consists of if-else conditions that assign the result based on the Marks and invoke this for every column row.

We can define multiple conditions for a column in a list and their corresponding values in another list if the condition is True . The select() method takes the list of conditions and their corresponding list of values as arguments and assigns them to the Result column.

A lambda is a small anonymous function consisting of a single expression. We will use lambda with apply() on the Marks column.

The x contains the marks in the lambda expression. We applied the if-else condition to the x and assigned the result accordingly in the Result column.

I am Fariba Laiq from Pakistan. An android app developer, technical content writer, and coding instructor. Writing has always been one of my passions. I love to learn, implement and convey my knowledge to others.

Related Article - Pandas Condition

• How to Create DataFrame Column Based on Given Condition in Pandas

DataFrames with Conditionals

The use of conditionals allows us to select a subset of rows based on the value in each row. Writing a conditional to select rows based on the data in a single column is straightforward and was used when we selected all of the courses taught by the Statistics department with the following code:

The subset of rows where the Subject is exactly equal to STAT (57 rows).

Complex Conditionals with Multiple Parts

As we want to answer more complex questions, we need increasingly complex conditionals. To help understand how a computer works, you may be familiar with the idea that computers ultimately only think in zeros and ones:

• When a computer stores a zero, we consider that to be False .
• When a computer stores a one, we consider that to be True .

When we use conditionals, we are assigning a truth value to every single row in the DataFrame.

• With our conditional df[df.Subject == "STAT"] , all rows where the Subject data was "STAT" was assigned a truth value of True and kept in the final result; all other rows were labeled False and discarded.

All programming languages allows us to combine conditionals together in two key ways: with an AND ( & ) or with an OR ( | ).

Multiple Conditionals Joined with AND ( & )

When we combine two conditionals, we can ask Python to keep only the result where the first conditional AND the second conditional are both True .

Writing a conditional with multiple parts requires the use of parenthesis around each individual conditional and an operation joining the two conditionals together. For example, using the Course Catalog dataset , we want all of the courses that are taught by Computer Science ( CS ) with a course number less than 300:

Both the first ( Subject is exactly equal to "CS" ) and second ( Number is less than 300 ) conditionals are checked independently. Since an AND ( & ) is used to join these two conditionals, the final truth value is True only when both conditionals are True :

All CS courses with course numbers less than 300 (17 rows).

Python allows us to continue to apply conditionals together infinitely long -- so it's no problem to have three conditionals:

All CS courses with course numbers less than 300 and exactly 3 credit hours (6 rows).

Multiple Conditionals Joined with OR ( | )

Alternatively, Python can combine two conditionals together and keep the result when either the first conditional OR the second conditional is True (this includes when they're both True as well!). There are two major applications when this is useful:

• Selecting multiple values of data from the same column (ex: all courses in "ARTD" OR "ARTE" OR "ARTF" ).
• Selecting multiple values from different columns and keeping all matches (ex: all courses in "PSYC" OR courses that are only 1 credit hour).

Selecting Multiple Values of Data from the Same Column

Looking at the first example above, the University of Illinois has a lot of courses in art across many different sub-areas of art including: Art Design ( "ARTD" ), Art Education ( "ARTE" ), Art Foundation ( "ARTF" ), Art History ( "ARTH" ), and Art Studio ( "ARTS" ).

To include ALL courses from all five sub-areas of art listed above, we must join them together with an OR ( | ). Notice that it is necessary to specify each conditional completely each time even though we are always comparing the subject since Python has to evaluate each conditional independently and then combine the results together:

All courses in any subjects ARTD, ARTE, ARTF, ARTH, OR ARTS (221 rows).

Selecting Multiple Values from Different Columns and Keeping All Matches

To be considered a "full-time student" at most universities, you must be enrolled in at least 12 credit hours . If you are only enrolled in 11 credit hours, you may be interested in any course that will bump you up to exactly 12 credit hours (ex: a course worth exactly one credit hour) or a course you may be interested in (ex: something from the psychology ( "PSYC" ) department).

To include ALL of the results of all courses that are either one credit hour OR in the psychology department, we need an OR :

All courses that are exactly one credit hour OR in the psychology department (490 rows).

Combining ANDs and ORs

The most complex conditionals will require a combination of both AND and OR statements. These can get incredibly tricky, but we can remember that Python will always process conditionals by only combining two conditionals together at a time.

Since Python combines only two conditionals together at any given time, it is critical we use parenthesis to ensure we specify the order that we want these conditionals combined. For example, let's explore only junior level (300-399) courses in Chemistry or Physics . To do so:

• The subject of the course must be CHEM or PHYS .
• The course number must be greater than or equal to 300 .
• The course number must also be less than 400 .

Naively writing this conditional results in the following code:

Default Order of Evaluation: AND before OR

If we do not use additional parenthesis, Python will always combine the ANDs first and then the ORs and will do so in left-to-right order. This means that:

The first set of two conditionals combined will be the first AND conditional: (df.Subject == "PHYS") & (df.Number >= 300) . The result contains all courses in PHYS with a number larger than 300.

The second set of two conditionals will be the result from #1 with the second AND : (Result of Step #1) & (df.Number < 400) . The result contains all courses in PHYS with a number from 300-399.

The final set of conditionals will be combined using OR : (df.Subject == "CHEM") | (Result of Step #2) . Since this is an OR , the result is ALL CHEM courses and then only the PHYS courses in the number 300-399.

We can verify our result by running the code:

The output of incorrect logic that does use parenthesis, which includes 500-level PHYS courses (92 rows).

Notice that the code appears correct until we scroll down ! The courses in Chemistry start at 300, but the last five rows show us that the courses in Physics include 500-level courses -- yikes!

Order of Evaluation: Using Parenthesis to Specify Order

Python uses parenthesis in a similar way to basic mathematics where the inner-most operations are done first. In our example, we want to make sure that all Chemistry and Physics courses are combined first, and only then can we limit the range of course numbers to the junior level.

By grouping both of these logical operations together, our new conditional can be thought of as a combination of two complex conditionals:

(df.Subject == "CHEM") | (df.Subject == "PHYS") , selecting only that are Chemistry OR Physics

(df.Number >= 300) & (df.Number < 400) , selecting only courses between 300 AND 399.

Joining these two conditionals together with an AND results in the exact output we expect:

All 300-level courses in chemistry or physics (11 rows).

Example Walk-Throughs with Worksheets

Video 1: dataframe conditionals using the party dataset.

• Download Blank Worksheet (PDF)

Video 2: DataFrame Conditionals using The Berkeley Dataset

Video 3: DataFrame Conditionals using The Course Catalog Dataset

Practice Questions

• Data Structure
• Coding Problems
• C Interview Programs
• C++ Aptitude
• Java Aptitude
• C# Aptitude
• PHP Aptitude
• Linux Aptitude
• DBMS Aptitude
• Networking Aptitude
• AI Aptitude
• MIS Executive
• Web Technologie MCQs
• CS Subjects MCQs
• Databases MCQs
• Programming MCQs
• Testing Software MCQs
• Digital Mktg Subjects MCQs
• Cloud Computing S/W MCQs
• Engineering Subjects MCQs
• Commerce MCQs
• More MCQs...
• Machine Learning/AI
• Operating System
• Computer Network
• Software Engineering
• Discrete Mathematics
• Digital Electronics
• Data Mining
• Embedded Systems
• Cryptography
• CS Fundamental
• More Tutorials...
• Tech Articles
• Code Examples
• Programmer's Calculator
• XML Sitemap Generator
• Tools & Generators

Home » Python » Python Programs

Vectorize conditional assignment in pandas dataframe

Given a pandas dataframe, we have to vectorize conditional assignment in pandas dataframe. By Pranit Sharma Last updated : October 03, 2023

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

Problem statement

We are given a DataFrame df with some columns and we want to create a new column based on some previous columns.

We want to apply some conditions like if the value of a column is less then some specific value then the value of a new column is some new specific value. If the value of that column is some other specific value then the value of the new column would be some new specific value and so on.

Vectorize conditional assignment

We will use pandas.DataFrame.loc property of pandas so that we can access the exact element that fits the condition and we can set the value of a new column for each value of the old column.

The pandas.DataFrame.loc property is a type of data selection method which takes the name of a row or column as a parameter. To perform various operations using the pandas.DataFrame.loc property, we need to pass the required condition of rows and columns in order to get the filtered data.

Let us understand with the help of an example,

Python program to vectorize conditional assignment in pandas dataframe

The output of the above program is:

Python Pandas Programs »

Related Tutorials

• Python - How to set column as date index?
• Seaborn: countplot() with frequencies
• SKLearn MinMaxScaler - scale specific columns only
• Pandas integer YYMMDD to datetime
• Select multiple ranges of columns in Pandas DataFrame
• Random Sample of a subset of a dataframe in Pandas
• Selecting last n columns and excluding last n columns in dataframe
• Search for a value anywhere in a pandas dataframe
• Pandas Number of Months Between Two Dates
• Pandas remove everything after a delimiter in a string
• Pandas difference between largest and smallest value within group
• Add a new row to a pandas dataframe with specific index name
• Sort dataframe by string length
• Pandas groupby for zero values
• Pandas dataframe remove all rows where None is the value in any column
• Missing data, insert rows in Pandas and fill with NAN
• Pandas: Output dataframe to csv with integers
• Pandas join dataframe with a force suffix
• Pandas DataFrame: How to query the closest datetime index?
• Sum of all the columns of a pandas dataframe with a wildcard name search

Comments and Discussions!

Load comments ↻

• Marketing MCQs
• Blockchain MCQs
• Artificial Intelligence MCQs
• Data Analytics & Visualization MCQs
• Python MCQs
• C++ Programs
• Python Programs
• Java Programs
• D.S. Programs
• Golang Programs
• C# Programs
• JavaScript Examples
• jQuery Examples
• CSS Examples
• C++ Tutorial
• Python Tutorial
• ML/AI Tutorial
• MIS Tutorial
• Software Engineering Tutorial
• Scala Tutorial
• Privacy policy
• Certificates
• Content Writers of the Month

Copyright © 2024 www.includehelp.com. All rights reserved.

Python Conditional Assignment

When you want to assign a value to a variable based on some condition, like if the condition is true then assign a value to the variable, else assign some other value to the variable, then you can use the conditional assignment operator.

In this tutorial, we will look at different ways to assign values to a variable based on some condition.

1. Using Ternary Operator

The ternary operator is very special operator in Python, it is used to assign a value to a variable based on some condition.

It goes like this:

Here, the value of variable will be value_if_true if the condition is true, else it will be value_if_false .

Let's see a code snippet to understand it better.

You can see we have conditionally assigned a value to variable c based on the condition a > b .

2. Using if-else statement

if-else statements are the core part of any programming language, they are used to execute a block of code based on some condition.

Using an if-else statement, we can assign a value to a variable based on the condition we provide.

Here is an example of replacing the above code snippet with the if-else statement.

3. Using Logical Short Circuit Evaluation

Logical short circuit evaluation is another way using which you can assign a value to a variable conditionally.

The format of logical short circuit evaluation is:

It looks similar to ternary operator, but it is not. Here the condition and value_if_true performs logical AND operation, if both are true then the value of variable will be value_if_true , or else it will be value_if_false .

Let's see an example:

But if we make condition True but value_if_true False (or 0 or None), then the value of variable will be value_if_false .

So, you can see that the value of c is 20 even though the condition a < b is True .

So, you should be careful while using logical short circuit evaluation.

While working with lists , we often need to check if a list is empty or not, and if it is empty then we need to assign some default value to it.

Let's see how we can do it using conditional assignment.

Here, we have assigned a default value to my_list if it is empty.

Assign a value to a variable conditionally based on the presence of an element in a list.

Now you know 3 different ways to assign a value to a variable conditionally. Any of these methods can be used to assign a value when there is a condition.

The cleanest and fastest way to conditional value assignment is the ternary operator .

if-else statement is recommended to use when you have to execute a block of code based on some condition.

Happy coding! 😊