Recently, I learn Python to download csv data from the Nasdaq Stock Screener page, https://www.nasdaq.com/market-activity/stocks/screener.
There are some ways to implement it. Here I just show one way. I will use time, datetime, and pandas packages to do it.
import time
import datetime as dt
import pandas as pd
from pandas.errors import EmptyDataError
# Debugging
import logging
log = logging.getLogger(__name__)
def read_all_stock_symbols():
df = pd.read_csv('assets/all-stock-symbols.csv', usecols=['Symbol'])
return df
def download_yahoo_symbol_csv():
all_symbols = read_all_stock_symbols()
start_date = int(time.mktime((dt.datetime.now() - dt.timedelta(days=1 * 365)).timetuple()))
end_date = int(time.mktime(dt.datetime.now().timetuple()))
interval = '1d' # 1d, 1m, 1yr
for index, row in all_symbols.iterrows():
symbol = row['Symbol']
api_url =
f'https://query1.finance.yahoo.com/v7/finance/download/{symbol}?'\
f'period1={start_date}&period2={end_date}&'\
f'interval={interval}&events=history&includeAdjustedClose=true'
try:
df = pd.read_csv(api_url)
print(symbol)
print(df)
except EmptyDataError as e:
log.debug('Cannot download: %s. Error details: %s', api_url, e)
print(e)
# Press the green button in the gutter to run the script.
if __name__ == '__main__':
download_yahoo_symbol_csv()
How Does It Work
pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language. pandas.read_csv is the key method to return DataFrame type. With the parameter [‘Symbol’], it will return the Symbol column data only.
To fetch the DataFrame type data, we can use its iterrows() method to yield both the index and row. The line 22 shows fetching each symbol and generate a URL, which can get each symbol’s historical data from the finance.yahoo.com.
To read the CSV, just use the read_csv() method to get DataFrame type data.
Error Handling
Generally, programmers cannot remember all kinds of exceptions that might occur in a complex operation, so they like to catch all exceptions. Therefore, it is also a good way to write undebugable code if you are not careful.
It is very important to record the actual cause of the exception somewhere for choosing to catch all exceptions(for example, log files, error messages printed to the screen, and so on)
Consider this example:
def parse_int(s):
try:
n = int(v)
except Exception:
print("Couldn't parse")
If you try this function, it behaves like this:
>>> parse_int(‘n/a’)
Couldn’t parse
>>> parse_int(’42’)
Couldn’t parse
>>>
The code example is from this book: Python Cookbook: Recipes for Mastering Python 3, 3rd Edition
Views: 15
