Project Background
Venturing into Data Science this past few weeks has exposed me to various tools and concepts that can transform how we think about data, process it, and utilize insights to make decisions or form conclusions about a specific variable or element. Seeing how application programming interface (API) work was enlightening on the tools available for retrieving and making sense of data.
The week's task was to extract data from https://www.football-data.org/ to determine the probabilities of each team in the Premier League winning the cup. The API nested in the site is a goldmine for football enthusiasts (I don't consider myself one) looking to scrape data and analyze matches and teams for various competitions across the major football leagues.
Libraries Imported
Tools utilized during this exercises included:
- Python
- Libraries
- pandas (data manipulation)
- requests (for API calls)
- python-dotenv for secure API key handling)
- matplotlib and seaborn (visualizations) and
- scipy.stats (handling probability distributions)
Data Extraction Process
The code kicked off with setting up the libraries
import requests
import os
from dotenv import load_dotenv
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats
Connecting to the /v4/competitions/PL/standings?season=2024 endpoint using the API key
Fetch data from API
response = requests.get(url, headers=headers)
data = response.json()
Probability Formula
def calculate_win_probability_poisson(wins, played, remaining):
if played == 0:
win_rate = 0.3
else:
win_rate = wins / (played + 1e-6)
expected_wins = win_rate * remaining + wins
prob = 1 - stats.poisson.cdf(24, expected_wins)
A Note on API Keys
A crucial insight gained from the exercise was the importance of using .env files to store API keys securely. These files are loaded via python-dotenv to enhance security by keeping sensitive data out of the public domain, where files are shared in public repositories.