Pandas – the data analysis and manipulation tool.
Data is often messy and rarely in perfect shape. This is especially true if the data comes from many different sources and the specifications are loosely defined. If you have access to data that is in great shape, it’s probably because someone else did the dirty work of validating it, cleaning it up, and normalizing …
So much useful data has a date or time component. Often, data has a timestamp to represent when the data was acquired, or when an event will take place, or as an identifying attribute like an expiration date. For this reason, understanding how to work with dates and times effectively can be a very useful …
Use pandas DateOffsets for easy date manipulation Read More »
Most pandas users encounter a situation where choosing to append rows to a pandas DataFrame seems like a good idea. A quick search of the API (or your favorite search engine) reveals that pandas has an append method in DataFrame. You may be tempted to use it. In this article I’ll show you why you …
There are big differences between overnight and intraday stock returns. This article shows how to use pandas to understand one big reason.
Indexing time series data in pandas is similar to other types, but there are a number of convenient functions unique to time series.
One of the most searched for (and discussed) questions about pandas is how to iterate over rows in a DataFrame. Often this question comes up right away for new users who have loaded some data into a DataFrame and now want to do something useful with it. The natural way for most programmers to think …
How to iterate over DataFrame rows (and should you?) Read More »
Removing one or more columns from a pandas DataFrame is a pretty common task, but it turns out there are a number of possible ways to perform this task. I found that this StackOverflow question, along with solutions and discussion in it raised a number of interesting topics. It is worth digging in a little bit to the …
How to remove a column from a DataFrame, with some extra detail Read More »
The query method in pandas DataFrames provides some flexibility in code, and potential speedups using numexpr.
This is the fifth post in a series on indexing and selecting in pandas. If you are jumping in the middle and want to get caught up, here’s what has been discussed so far: Basic indexing, selecting by label and location Slicing in pandas Selecting by boolean indexing Selecting by callable Once the basics were …
In pandas, you can use callables where indexers are accepted. It turns out that can be handy for a pretty common use case.