A few months ago I challenged myself to develop a simple Pre-market scanner without spending any money on subscriptions or APIs. I was able to do this using the web scraping technique. I blogged about this on How to Code your own Pre-market Stocks Scanner Web application (Using Python). Aside from Benzinga, I found another website that displays a list of highest pre-market gainers, TheStockCatalyst.com. It displays the recent headlines to hopefully give the viewers “an indication why the stock is moving“.
However, sometimes there is no Recent Headlines from TheStockCatalyst.com causing the scanner results to display no news. I had to manually search the web to find out the reason of price increase.
In order to fix this, I did my research on where to get real-time stock market news for stock tickers programmatically. There are free REST API endpoints and there are also paid API subscriptions that offers core services such as Market Data, Historical Data, etc. and they include News API endpoint(s).
Free Resources
| Resource | Sample Link | Notes |
| Google News | https://news.google.com/rss/search?q=AAPL | Write a program that will consume RSS feed, sort and filter the results. |
| Yahoo Finance | https://query2.finance.yahoo.com/v1/finance/search?q=AAPL | Extract the news element in the JSON result. Other Yahoo Finance endpoints can be found by looking at the source codes of popular Python library YFinance. |
| Bing News Search | Documentation is in https://docs.microsoft.com/en-us/azure/cognitive-services/bing-news-search/search-the-web | |
| https://twitter.com/search?q=%24AAPL | Most tweets may not be relevant. Parsing the results may be difficult. Twitter API may be worth looking at. |
Stock Market API Paid Subscriptions
If you want to avail of paid subscription(s), the subscription fee should not be the top criteria. It is also important that the news that the API shows should be relevant and current.
Code for Getting News List for a Ticker (Using Alpaca REST API):
Since I already subscribed to Alpaca, I use their endpoint to retrieve the latest news of a certain tickers.
import alpaca_trade_api as tradeapi
import os
API_KEY = os.environ.get('APCA_API_KEY_ID')
API_SECRET_KEY = os.environ.get('APCA_API_SECRET_KEY')
ticker = 'AAPL'
api = tradeapi.REST(API_KEY, API_SECRET_KEY, NEWS_URL, api_version='v2')
news_list = api.get_news(ticker
news_link = ''
for news_ctr in range(len(news_list)):
news_obj = news_list[news_ctr]
news_link += f"<a href=\'{news_obj.url}\' target='_blank'>{news_obj.headline} [{news_obj.updated_at}]</a><br/><br/>"
Here is the JSON result when I retrieve the news about AAPL
[NewsV2({ 'author': 'Maureen Meehan',
'content': '',
'created_at': '2022-09-01T18:30:21Z',
'headline': 'EXCLUSIVE: Outstanding Cannabis Podcasts, Meet The Producers '
'& Guests At Benzinga Cannabis Capital Conference',
'id': 28723850,
'images': [ { 'size': 'large',
'url': 'https://cdn.benzinga.com/files/imagecache/2048x1536xUP/images/story/2022/09/01/cannabis_podcast_irina_popova_st_by_shutterstock.jpg'},
{ 'size': 'small',
'url': 'https://cdn.benzinga.com/files/imagecache/1024x768xUP/images/story/2022/09/01/cannabis_podcast_irina_popova_st_by_shutterstock.jpg'},
{ 'size': 'thumb',
'url': 'https://cdn.benzinga.com/files/imagecache/250x187xUP/images/story/2022/09/01/cannabis_podcast_irina_popova_st_by_shutterstock.jpg'}],
'source': 'benzinga',
'summary': 'A stroll through Spotify\xa0(NYSE: SPOT) and Apple (NASDAQ: '
'AAPL) iTunes\xa0reveals that there are over 2000 '
'cannabis-related podcasts.\xa0',
'symbols': ['AAPL', 'CURLF', 'GHBWF', 'GRAMF', 'SPOT', 'TCNNF'],
'updated_at': '2022-09-01T20:45:46Z',
'url': 'https://www.benzinga.com/markets/cannabis/22/09/28723850/outstanding-cannabis-podcasts-meet-the-producers-guests-at-benzinga-cannabis-capital-conference'}),
NewsV2({ 'author': 'Michael Horton',
'content': '',
'created_at': '2022-09-01T18:03:17Z',
'headline': 'Top 15 Trending Stocks On WallStreetBets As Of Thursday, '
'September 1, 2022 (Via Swaggy Stocks)',
'id': 28723480,
'images': [],
'source': 'benzinga',
'summary': '',
'symbols': [ 'AAPL',
'AI',
'AMD',
'AMZN',
'BBBY',
'GME',
'INTC',
'LULU',
'META',
'NVDA',
'OKTA',
'SNAP',
'SQQQ',
'TQQQ',
'TSLA'],
'updated_at': '2022-09-01T18:03:18Z',
'url': 'https://www.benzinga.com/trading-ideas/22/09/28723480/top-15-trending-stocks-on-wallstreetbets-as-of-thursday-september-1-2022-via-swaggy-stocks'})
]Here is the actual HTML output of the above Python code:
Finally, after implementing the Pyton code on Pre-market scanner here is a screenshot of the fixed results page. The missing entries in News column were populated from the Alpaca News API.
Based on my observation, Alpaca’s news are all sourced from Benzinga. I then checked out the pricing and threshold calls of Benzinga’s APIs if I want to call their endpoints directly rather than going through Alpaca. But I only found this:
To get started with Benzinga’s APIs please email us directly at licensing@benzinga.com or give us a call: 877-440-9464. We will respond within a business day.
For now, I will stick to using Alpaca API news endpoint as it solves my problem. If there’s a need for better source of news, I will revisit this research again.
