Pandas is great for dealing with both numerical and text data. In most projects you'll need to clean up and verify your data before analysing or using it for anything useful. Data might be delivered in databases, csv or other formats of data file, web scraping results, or even manually entered. Once you have loaded … Continue reading Converting types in Pandas
Tag: python
You can easily and sensibly run multiple versions of Python with pyenv
Python 3.9 just came out recently, and I thought it would make sense to check out some of the new features (dict union operators, string remove prefix and suffix, etc.). Of course, doing this requires a Python 3.9 environment. Since new versions of Python may break existing code, I don't want to update my entire … Continue reading You can easily and sensibly run multiple versions of Python with pyenv
Python: how to use multiprocessing to finish work faster
It's very common in data science work to have some function or a set of functions that you run in a loop to process or analyze data. When you see a loop that's performing an expensive operation, you should immediately think of (at least) two ways to speed things up. The first is vectorization, which … Continue reading Python: how to use multiprocessing to finish work faster
Basic Pandas: Moving a DataFrame column
Sometimes we want to change the ordering of a DataFrame's columns. Maybe you are saving your data and require the columns to be in a certain order, or you may have a large number of columns and want a few of them to be visible whenever viewing your DataFrame in an interactive session, so you … Continue reading Basic Pandas: Moving a DataFrame column
Basic Pandas: Renaming a DataFrame column
A very common need in working with pandas DataFrames is to rename a column. Maybe the columns were supplied by a data source like a CSV file and they need cleanup. Or maybe you just changed your mind during an interactive session. Let's look at how you can do this, because there's more than one … Continue reading Basic Pandas: Renaming a DataFrame column
Basic Pandas: How to add a column to a DataFrame
Pandas is one of my favorite Python libraries, and I use it every day. A very common action is to add a column to a DataFrame. This is a pretty basic task. I'm going to look at a few examples to better show what is happening when we add a column, and how we need … Continue reading Basic Pandas: How to add a column to a DataFrame
How to connect to Interactive Brokers using Python
The discount brokerage space is getting very competitive with commissions going to zero recently at many brokerages, including Interactive Brokers. IB has long been a broker with one of the largest breadth of products and service offerings targeting a professional audience. It is also a low cost brokerage making it an excellent option for the … Continue reading How to connect to Interactive Brokers using Python
3 ways to get historical market data from IEX Cloud
IEX Cloud is a service offering financial data. They offer a wide variety of data sets, both for historical and real-time data. Those of us interested in financial data, whether for trading or research, are always looking for good data sources, especially for clean and complete data. This post will show you three ways to … Continue reading 3 ways to get historical market data from IEX Cloud
Python string literals
I've been working on moving a codebase from python 2 to 3 recently and decided to take a fresh look at string literals. In python, string literals can be enclosed by a pair of unescaped single quotes ', a pair of double quotes ", or a pair of triple double quotes """ or single quotes … Continue reading Python string literals
A pandas.DataFrame.apply example
I recently saw a question about pandas.DataFrame.apply and realized that when I first started using Pandas I would often attempt to solve problems with apply when a vectorized solution was what I should have been using instead. Let's say that you have an existing function to calculate the present value of an investment that takes … Continue reading A pandas.DataFrame.apply example