AI & Data

Working with REST APIs

Module 3

Fetching and integrating data from web APIs for machine learning and data analysis projects

API GET JSON Python Client

Introduction to Web APIs

Key insight: APIs let you access data from thousands of services - weather, finance, social media, and more!

REST Request/Response Flow

Here's what happens when you call an API:

HTTP Request → JSON Response → DataFrame
Python Script REST API Database requests.get(url) GET /users Query users User records JSON (200 OK) response.json() Parse JSON → dict Create DataFrame dict → DataFrame

How REST APIs Work

MethodActionExample
GETRetrieve dataGet all users
POSTCreate new dataAdd a user
PUTUpdate existing dataUpdate user info
DELETERemove dataDelete a user

Our focus will be on GET for data retrieval.

Making Your First API Request

We use the requests library in Python:

import requests

# Define the API endpoint URL
url = "https://jsonplaceholder.typicode.com/posts/1"

# Send a GET request
response = requests.get(url)

# The response object contains server's response
print(response.status_code)  # e.g., 200
print(response.json())       # The data in JSON format

Understanding the Response

HTTP Status Codes:

200 OKThe request was successful
404 Not FoundResource could not be found
401 UnauthorizedAuthentication required
403 ForbiddenNo permission to access
500 Server ErrorProblem on the server

Response Body: The actual data, often in JSON format (key-value pairs).

Exercise 1: Simple GET Request

Goal: Fetch a list of users from a public API

API Endpoint: https://jsonplaceholder.typicode.com/users

  1. Write a Python script using requests
  2. Make a GET request to the endpoint
  3. Check if the request was successful (status code 200)
  4. Print the name of the first user in the list
Tip: Remember to install requests first: pip install requests

Working with API Parameters

Path Parameters

Part of the URL path to identify a specific resource:

/users/1

Gets user with ID 1

Query Parameters

Key-value pairs to filter or sort:

/posts?userId=1

Gets posts by user 1

Passing query parameters with requests:

params = {'userId': 1}
response = requests.get('.../posts', params=params)
print(response.url)  # See the final URL

Exercise 2: Filtering API Results

Goal: Fetch comments for a specific post

API Endpoint: https://jsonplaceholder.typicode.com/comments

  1. Use query parameters to get comments for postId=1
  2. Write a Python script to perform this request
  3. Print the number of comments received
  4. Print the email of the first commenter

Handling Authentication

Why? To identify the client, for security, and for rate limiting.

Common methods:

Sending a key in the headers:

headers = {'Authorization': 'Bearer YOUR_API_KEY'}
response = requests.get(url, headers=headers)
Security: Never commit API keys to version control!

Exercise 3: Error Handling

Goal: Write robust code that handles potential API errors

  1. Request a non-existent post (e.g., /posts/99999)
  2. Check the status_code to see if it's 200
  3. If not 200, print an informative error message
  4. Wrap the request in a try...except block to catch network errors
try:
    response = requests.get(url, timeout=5)
    response.raise_for_status()  # Raises exception for bad status
except requests.RequestException as e:
    print(f"Request failed: {e}")

Integrating with Orange Data Mining

Goal: Convert API data into an Orange Data Table

from Orange.data import Table, Domain, StringVariable
import requests

# 1. Fetch data from API
users_data = requests.get(
    'https://jsonplaceholder.typicode.com/users'
).json()

# 2. Define the domain (the columns)
domain = Domain([
    StringVariable('name'),
    StringVariable('email'),
    StringVariable('city')
])

# 3. Create the data table
data = [[u['name'], u['email'], u['address']['city']]
        for u in users_data]
out_data = Table.from_list(domain, data)

Final Exercise: World Countries Data

API: RestCountries - https://restcountries.com/v3.1/all

Complete REST → Orange Pipeline
REST Countries API GET /v3.1/all 250+ countries Python Processing Fetch JSON requests.get() Extract fields name, region population, area Create Domain Orange.data domain = Domain([StringVariable('name'), ContinuousVariable('population')]) Orange Visualization Data Table View all data GeoMap World view!

Questions?

Working with REST APIs

Next: Web Scraping Techniques

Slide Overview