A common use case in pandas is to want to apply a function to rows in a DataFrame. For a novice, the temptation can be to iterate through the rows in the DataFrame and pass the data to a function, but that is not a good idea. (You can read this article for a detailed …
Options to run pandas DataFrame.apply in parallel Read More »
Data is often messy and rarely in perfect shape. This is especially true if the data comes from many different sources and the specifications are loosely defined. If you have access to data that is in great shape, it’s probably because someone else did the dirty work of validating it, cleaning it up, and normalizing …
A stock index is just a list of stocks. But an index is a special list because investors use it to make investing decisions. An index is constructed via rules about stocks to include, how much to include, and when to include (or remove it). Finding this data, especially for more obscure indexes, can be …
Finding and analyzing free stock index data with Python and EDGAR Read More »
So much useful data has a date or time component. Often, data has a timestamp to represent when the data was acquired, or when an event will take place, or as an identifying attribute like an expiration date. For this reason, understanding how to work with dates and times effectively can be a very useful …
Use pandas DateOffsets for easy date manipulation Read More »
Most pandas users encounter a situation where choosing to append rows to a pandas DataFrame seems like a good idea. A quick search of the API (or your favorite search engine) reveals that pandas has an append method in DataFrame. You may be tempted to use it. In this article I’ll show you why you …
If you’ve used a Jupyter notebook, you’ve used a kernel. A kernel is a process that executes code from a front-end process. Usually, if you are working in Python, the kernel you use is the IPython kernel. The IPython kernel usually matches the Python version and contains the same libraries as the process running the …
Some sources of financial data can be expensive or difficult to find. For example, some is only available from exchanges or vendors who charge a hefty fee for access. However, the financial industry is also heavily regulated, and one of its main regulators provides free access to its data. The U.S. Securities and Exchange Commission …
An introduction to accessing financial data in EDGAR, using Python Read More »
Argparse doesn’t support date arguments by default, but it can be easily extended to parse and validate dates in your Python scripts.
Sometimes pandas.read_html doesn’t work for scraping website data, but you can try using requests and BeautifulSoup to do it yourself.
There are big differences between overnight and intraday stock returns. This article shows how to use pandas to understand one big reason.