Pandas in Python: Complete Beginner Guide

Pandas in Python is one of the most widely used tools for working with structured data. It helps you load spreadsheets and CSV files, inspect tables, clean missing values, filter rows, group records, combine datasets, and export results with readable code.

This beginner guide introduces the two core objects—Series and DataFrame—and builds a practical workflow from importing data to producing a summary report. You do not need previous data science experience, but basic knowledge of Python data types, lists, dictionaries, and functions will help.

What Is Pandas?

Pandas is an open-source Python library designed for labeled and tabular data. A DataFrame resembles a spreadsheet or database table: rows represent observations, columns represent variables, and each column has a name and data type.

The official Pandas documentation contains the full reference. The official 10 minutes to pandas guide is a useful next step after this tutorial.

Install Pandas

python -m pip install pandas

For Excel files, also install an engine such as openpyxl:

python -m pip install openpyxl

Use a virtual environment so each project has isolated dependencies.

Import Pandas

import pandas as pd

print(pd.__version__)

The alias pd is a community convention used in most examples and documentation.

Create a Series

A Series is a one-dimensional labeled collection:

import pandas as pd

prices = pd.Series(
    [19.90, 35.50, 12.00],
    index=["book", "course", "template"],
    name="price",
)

print(prices)
print(prices["course"])

Series objects are often encountered as individual DataFrame columns.

Create a DataFrame

import pandas as pd

data = {
    "product": ["Book", "Course", "Template"],
    "quantity": [3, 2, 5],
    "unit_price": [19.90, 35.50, 12.00],
}

sales = pd.DataFrame(data)
print(sales)

A dictionary is a convenient way to create a small table because each key becomes a column.

Inspect a DataFrame

print(sales.head())
print(sales.tail())
print(sales.shape)
print(sales.columns)
print(sales.dtypes)
sales.info()

• head() and tail() preview rows.
• shape returns rows and columns.
• dtypes shows each column type.
• info() summarizes memory, types, and non-null values.

Inspecting the table before transforming it prevents many mistakes.

Read a CSV File

sales = pd.read_csv(
    "sales.csv",
    encoding="utf-8",
)

print(sales.head())

CSV files may use different delimiters:

sales = pd.read_csv(
    "sales-semicolon.csv",
    sep=";",
    encoding="utf-8",
)

The Python CSV guide compares Pandas with the standard csv module.

Read an Excel File

sales = pd.read_excel(
    "sales.xlsx",
    sheet_name="Sales",
)

Use usecols to load only relevant columns and parse_dates for date columns:

sales = pd.read_excel(
    "sales.xlsx",
    sheet_name="Sales",
    usecols=["Date", "Product", "Quantity", "Unit Price"],
    parse_dates=["Date"],
)

See the Excel import guide for workbook-specific techniques.

Select Columns

products = sales["product"]
subset = sales[["product", "quantity"]]

One pair of brackets returns a Series. A list of column names inside double brackets returns a DataFrame.

Select Rows with loc and iloc

loc uses labels and conditions. iloc uses integer positions.

first_row = sales.iloc[0]
first_three = sales.iloc[:3]

selected = sales.loc[
    sales["quantity"] >= 3,
    ["product", "quantity"],
]

Use explicit column selection to make the result easier to understand.

Filter Rows

expensive = sales[sales["unit_price"] > 20]

selected_products = sales[
    sales["product"].isin(["Book", "Course"])
]

range_filter = sales[
    sales["unit_price"].between(10, 30)
]

Combine conditions with & for AND and | for OR. Wrap each condition in parentheses:

result = sales[
    (sales["quantity"] >= 2)
    & (sales["unit_price"] < 30)
]

Create and Modify Columns

sales["total"] = sales["quantity"] * sales["unit_price"]
sales["product_upper"] = sales["product"].str.upper()

Most Pandas operations work on entire columns without an explicit Python loop. This vectorized style is often clearer and faster.

Clean Column Names

sales.columns = (
    sales.columns
    .str.strip()
    .str.lower()
    .str.replace(" ", "_", regex=False)
)

Consistent labels such as unit_price are easier to type and reference than labels containing spaces.

Work with Missing Values

print(sales.isna().sum())

Remove rows missing critical fields:

sales = sales.dropna(subset=["product", "quantity"])

Fill missing values when a replacement is meaningful:

sales["quantity"] = sales["quantity"].fillna(0)
sales["unit_price"] = sales["unit_price"].fillna(
    sales["unit_price"].median()
)

Do not fill missing data blindly. The correct decision depends on why the value is absent.

Convert Data Types

sales["quantity"] = pd.to_numeric(
    sales["quantity"],
    errors="coerce",
)

sales["date"] = pd.to_datetime(
    sales["date"],
    errors="coerce",
)

Invalid values become missing values with errors="coerce", allowing you to review or remove them later.

Sort Data

sales = sales.sort_values(
    by=["total", "product"],
    ascending=[False, True],
)

Unlike list.sort(), sort_values() returns a new DataFrame unless you assign the result or request an in-place change.

Group and Aggregate

summary = (
    sales.groupby("product", as_index=False)
    .agg(
        units=("quantity", "sum"),
        revenue=("total", "sum"),
        average_price=("unit_price", "mean"),
    )
    .sort_values("revenue", ascending=False)
)

print(summary)

groupby() follows a split-apply-combine pattern: split rows into groups, apply calculations, and combine the results.

Count Values

print(sales["product"].value_counts())
print(sales["product"].nunique())

value_counts() produces a frequency table. nunique() counts distinct values.

Merge Two DataFrames

products = pd.DataFrame({
    "product_id": [1, 2, 3],
    "category": ["Books", "Courses", "Templates"],
})

orders = pd.DataFrame({
    "order_id": [101, 102, 103],
    "product_id": [1, 3, 1],
    "quantity": [2, 5, 1],
})

combined = orders.merge(
    products,
    on="product_id",
    how="left",
)

print(combined)

A left merge keeps every row from the orders table and adds matching product information.

Concatenate Tables

all_sales = pd.concat(
    [january_sales, february_sales],
    ignore_index=True,
)

Concatenation stacks compatible tables. Confirm that column names and types match before combining them.

Work with Dates

sales["year"] = sales["date"].dt.year
sales["month"] = sales["date"].dt.month
sales["weekday"] = sales["date"].dt.day_name()

The datetime guide explains parsing and date concepts beyond Pandas.

Export Results

summary.to_csv(
    "sales-summary.csv",
    index=False,
    encoding="utf-8",
)

summary.to_excel(
    "sales-summary.xlsx",
    index=False,
)

Setting index=False prevents Pandas from writing the DataFrame index as an extra column.

A Complete Beginner Project

from pathlib import Path
import pandas as pd

INPUT = Path("data") / "sales.csv"
OUTPUT = Path("output") / "product-summary.csv"


def load_data(path: Path) -> pd.DataFrame:
    return pd.read_csv(path, encoding="utf-8")


def clean_data(data: pd.DataFrame) -> pd.DataFrame:
    cleaned = data.copy()
    cleaned.columns = (
        cleaned.columns
        .str.strip()
        .str.lower()
        .str.replace(" ", "_", regex=False)
    )
    cleaned["product"] = cleaned["product"].astype("string").str.strip()
    cleaned["quantity"] = pd.to_numeric(cleaned["quantity"], errors="coerce")
    cleaned["unit_price"] = pd.to_numeric(cleaned["unit_price"], errors="coerce")
    cleaned = cleaned.dropna(
        subset=["product", "quantity", "unit_price"]
    )
    cleaned["total"] = cleaned["quantity"] * cleaned["unit_price"]
    return cleaned


def summarize(data: pd.DataFrame) -> pd.DataFrame:
    return (
        data.groupby("product", as_index=False)
        .agg(
            units=("quantity", "sum"),
            revenue=("total", "sum"),
        )
        .sort_values("revenue", ascending=False)
    )


def main() -> None:
    if not INPUT.exists():
        raise FileNotFoundError(f"Missing input file: {INPUT}")

    OUTPUT.parent.mkdir(parents=True, exist_ok=True)
    data = clean_data(load_data(INPUT))
    report = summarize(data)
    report.to_csv(OUTPUT, index=False, encoding="utf-8")
    print(f"Saved report to {OUTPUT}")


if __name__ == "__main__":
    main()

This structure separates input, cleaning, aggregation, and output. It is easier to test and maintain than placing every operation in one large block. Use the pathlib guide for file paths and Pytest for automated checks.

Avoid SettingWithCopy Confusion

When creating a filtered DataFrame that you plan to modify, make an explicit copy:

high_value = sales.loc[sales["total"] > 1000].copy()
high_value["priority"] = True

This makes your intention clear and prevents ambiguous chained assignments.

Common Beginner Mistakes

• Skipping head(), info(), and type inspection.
• Assuming numeric-looking columns are actually numeric.
• Using Python loops where a vectorized column operation is clearer.
• Filling all missing values with zero without understanding them.
• Forgetting parentheses around combined filter conditions.
• Using inplace=True everywhere instead of assigning readable results.
• Saving CSV or Excel output with an unwanted index column.
• Modifying a filtered view without an explicit copy.

Frequently Asked Questions

Is Pandas included with Python?

No. Install it with pip, Conda, Poetry, or another package manager.

Is Pandas only for data science?

No. It is also useful for reports, spreadsheet automation, data migration, validation, and operational scripts.

Can Pandas handle millions of rows?

It can handle large datasets when enough memory is available, but very large files may require chunked processing, optimized types, or another tool.

Should I use Pandas for every CSV file?

No. The standard csv module may be simpler for small streaming tasks. Pandas is valuable when you need table transformations and analysis.

Conclusion

Pandas makes tabular data easier to inspect, clean, transform, summarize, merge, and export. A dependable workflow begins by understanding the input: preview rows, check column names, inspect types, and count missing values before changing anything.

Start with a small CSV or Excel file and reproduce the complete project. Then add one transformation at a time and verify the result. Careful inspection is more important than writing the shortest possible chain of methods.