Pandas is great for dealing with both numerical and text data. In most projects you’ll need to clean up and verify your data before analysing or using it for anything useful. Data might be delivered in databases, csv or other formats of data file, web scraping results, or even manually entered. Once you have loaded …
Python 3.9 just came out recently, and I thought it would make sense to check out some of the new features (dict union operators, string remove prefix and suffix, etc.). Of course, doing this requires a Python 3.9 environment. Since new versions of Python may break existing code, I don’t want to update my entire …
You can easily and sensibly run multiple versions of Python with pyenv Read More »
It’s very common in data science work to have some function or a set of functions that you run in a loop to process or analyze data. When you see a loop that’s performing an expensive operation, you should immediately think of (at least) two ways to speed things up. The first is vectorization, which …
Python: how to use multiprocessing to finish work faster Read More »
Sometimes we want to change the ordering of a DataFrame’s columns. Maybe you are saving your data and require the columns to be in a certain order, or you may have a large number of columns and want a few of them to be visible whenever viewing your DataFrame in an interactive session, so you …
A very common need in working with pandas DataFrames is to rename a column. Maybe the columns were supplied by a data source like a CSV file and they need cleanup. Or maybe you just changed your mind during an interactive session. Let’s look at how you can do this, because there’s more than one …
Pandas is one of my favorite Python libraries, and I use it every day. A very common action is to add a column to a DataFrame. This is a pretty basic task. I’m going to look at a few examples to better show what is happening when we add a column, and how we need …
Basic Pandas: How to add a column to a DataFrame Read More »
The discount brokerage space is getting very competitive with commissions going to zero recently at many brokerages, including Interactive Brokers. IB has long been a broker with one of the largest breadth of products and service offerings targeting a professional audience. IB is also a low cost brokerage, so that makes it a good option …
How to connect to Interactive Brokers using Python Read More »
IEX Cloud is a service offering financial data. They offer a wide variety of data sets, both for historical and real-time data. Those of us interested in financial data, whether for trading or research, are always looking for good data sources, especially for clean and complete data. This post will show you three ways to …
3 ways to get historical market data from IEX Cloud Read More »
I’ve been working on moving a codebase from python 2 to 3 recently and decided to take a fresh look at string literals. In python, string literals can be enclosed by a pair of unescaped single quotes ‘, a pair of double quotes “, or a pair of triple double quotes “”” or single quotes …
I recently saw a question about pandas.DataFrame.apply and realized that when I first started using Pandas I would often attempt to solve problems with apply when a vectorized solution was what I should have been using instead. Let’s say that you have an existing function to calculate the present value of an investment that takes …