Python Requests Tutorial: How to Call APIs in Python

1. Introduction

• Apply best practices which are used by senior engineers in real systems 

2. What is an API? (Simple Explanation with Real Example)

An API (Application Programming Interface) is a structured contract that allows different software systems to communicate with each other in a standardized and predictable way.

In simple words, an API is a bridge between two systems — one system sends a request and the other system sends back a response.

Instead of directly accessing databases or internal logic, applications interact through APIs to ensure security, scalability, and loose coupling.

Why This Matters in Real Systems

In real-world systems, you don’t build applications as a single block.

They are divided into multiple services like:

• user experience 

• payment service 

• notification service 

Finally, all these services need to communicate with each other as well.

If you aren’t going to use APIs, then:

• services can’t exchange data 

• systems would become tightly coupled 

• microservices architecture breaks down 

Due to this, APIs are the backbone of modern systems.

Real Example

Let us try to understand this with a simple payment system.

When a user makes an online payment, then your system doesn’t handle everything internally.

Instead, it calls multiple APIs to handle different tasks:

• Payment API: It verifies and processes the transaction 

• Fraud API: It checks if the transaction is safe or not 

• Database API: It stores your transaction details in the database 

Overall, the complete flow looks like below:

 

This flow shows how multiple APIs work together in real-world systems to process a single user request.

As you can see, each step depends on the API response.

What Actually Happens (Behind the Scenes)

When you call an API:

• Your application sends a request (URL + data) 

• Server processes the request 

• Server sends back a response (usually JSON) 

3. What is the Python Requests Library?

The Python Requests library is a high-level HTTP client that simplifies API communication by allowing your Python application to send requests and receive responses in a clean and readable way.

Why This Matters

This becomes especially important when building real-world applications where reliability, performance, and simplicity matter.

If you use low-level libraries like socket, http.client or even urllib, then you need to manually handle multiple things like:

• socket connections 

• request formatting 

• response parsing 

It would also increase complexity and chances of errors.

On the other side, the Requests library solves this problem and provides multiple benefits like:

• abstracting low-level networking 

• reducing boilerplate (repetitive) code 

• improving developer productivity 

So, you don’t need to write lengthy networking code (of 20–30 lines), you only need to write just one line.

Example: Basic API Call

import requests
response = requests.get("https://api.github.com")
print(response.status_code)

Explanation

import requests

requests.get("https://api.github.com")

Using urllib:

from urllib import request

request.urlopen("https://api.github.com")

Explanation:

pip install requests

This command installs the Requests library using pip (Python package manager) globally or in your current environment.

python -m venv venv

venv is the folder name where all dependencies will be stored.

source venv/bin/activate

Windows:

venv\Scripts\activate

Once a virtual environment is activated, then your terminal is now inside the virtual environment.

pip install requests

It would install the Requests library and this installation is now isolated and it won’t affect other projects.

import requests
print(requests.__version__)

Explanation:

import requests

response = requests.get("https://api.github.com")

print(response.status_code)
print(response.json())

Explanation:

if response.status_code == 200:
    data = response.json()

Your API can fail anytime due to various reasons like network issues, server downtime, or invalid requests.

7. Understanding HTTP Methods

Common HTTP Methods

import requests

# GET
requests.get("https://api.example.com/users")

# POST
requests.post("https://api.example.com/users", json={"name": "Amit"})
# PUT
requests.put("https://api.example.com/users/1", json={"name": "Amit updated"})
# DELETE
requests.delete("https://api.example.com/users/1")

Explanation:

import requests

params = {
    "q": "Python",
    "sort": "stars"
}

response = requests.get("https://api.github.com/search/repositories", params=params)

print(response.json())

Explanation:

9. How to Send POST Requests with JSON Data

import requests

data = {
    "name": "Amit",
    "role": "AI Architect"
}

response = requests.post("https://api.example.com/users", json=data)

print(response.status_code)
print(response.json())

Explanation:

import requests

response = requests.get("https://api.github.com")

print(response.status_code)
print(response.headers)
print(response.text)

Explanation:

11. Handling Errors and Exceptions in API Calls

a) Handling Timeout:

response = requests.get(url, timeout=5)

Explanation:

b) Using Try-Except (Safe Execution)

try:
    response = requests.get(url, timeout=5)
    response.raise_for_status()
    data = response.json()

except requests.exceptions.RequestException as e:
    logging.error(f"API Error: {e}")
    data = None

Explanation:

if response.status_code == 200:
    data = response.json()
else:
    print("Invalid response")

Explanation:

import requests
import os
url = "https://api.example.com/data"
# Get secrets from environment variables (secure way)
api_key = os.getenv("API_KEY")
token = os.getenv("API_TOKEN")

headers = {
    "x-api-key": api_key,
    "Authorization": f"Bearer {token}"
}
try:
    response = requests.get(url, headers=headers, timeout=5)
    response.raise_for_status()
    data = response.json()
    print(data)

except requests.exceptions.RequestException as e:
    logging.error(f"API Error: {e}")

Explanation:

import requests

url = "https://api.openweathermap.org/data/2.5/weather"

params = {
    "q": "Delhi",
    "appid": "YOUR_API_KEY",
    "units": "metric"
}

try:
    response = requests.get(url, params=params, timeout=5)
    response.raise_for_status()
    data = response.json()

    print("City:", data["name"])
    print("Temperature:", data["main"]["temp"], "°C")
    print("Weather:", data["weather"][0]["description"])

except requests.exceptions.RequestException as e:
    logging.error(f"API Error: {e}")

Explanation:

14. Best Practices (How to Build Systems Correctly)

If you would follow these best practices, then it would make your system stable.

15. Production-Level API Integration in Python

Introduction

In real-world applications, API integration is not just about calling an endpoint. It involves handling failures, validating data, retrying intelligently, and organizing code in a structured way.

In this guide, we will build a complete production-level API system step by step so that even beginners can understand how real engineers design scalable and reliable applications.

Problem: Without Proper Design

Before building the system, let’s first understand the problem with basic API calls.

In many beginner examples, APIs are called directly without handling errors or failures. This works for small tests but breaks in real systems.

Let’s see a simple example.

import requests

response = requests.get("https://api.example.com/data")

print(response.json())

Explanation

This code looks simply, but it is not suitable for production use. It directly calls the API without checking if the request fails, times out, or returns invalid data. If anything goes wrong, the system can crash or behave unexpectedly.

There is no retry logic, no timeout handling, and no validation. Everything is hardcoded, which makes it difficult to maintain and scale. In real-world applications, this approach is risky and should be avoided.

Let us now visualize how a production-level API system is structured before writing the code.

Project Structure

Before writing code, we organize the project so that each file has a clear responsibility.

Instead of writing everything in one file, we divide the code into folders like config, client, service, and validation. 

This structure is built using core concepts of Python modules and packages, which allow developers to organize code into reusable and scalable components in real-world applications.

This makes the system clean and scalable.

Below is the final production-level structure.

Explanation

In this structure, we separate code based on responsibility so that each part of the system does one job. Config handles environment values, clients manage API calls, and core contains shared logic like logging and exceptions.

We also have models for structured data, validators for checking responses, and services for business logic. The __init__.py files make each folder a module so imports work properly. This structure makes the system clean, scalable, and easy to understand.

Production-Level API Integration Flow:

Explanation:

This diagram shows how data flows through the system from start to end. The request starts from main.py, goes through the service layer, and then reaches the API client where the actual API call is made.

Once the response is received, it is validated and converted into a structured model before returning the final output. Config, logging, and exception handling work in the background to make the system reliable and production-ready.

What is __init__.py?

# __init__.py (empty file)

Explanation

import os

BASE_URL = os.getenv("API_BASE_URL", "https://api.github.com")
TIMEOUT = int(os.getenv("API_TIMEOUT", 5))
RETRIES = int(os.getenv("API_RETRIES", 3))
RETRY_DELAY = int(os.getenv("API_RETRY_DELAY", 2))

Explanation

In this code, we are defining configuration values using environment variables. Instead of hardcoding values inside the code, we use os.getenv() to read them dynamically from the system.

If the environment variable is not set, Python uses the default value. This makes the system flexible and safe for both development and production. In simple terms, this allows us to change behavior without modifying the code.

2. Logging (logger.py)

In production systems, it is important to track what is happening inside the application.

Instead of using print statements, we use logging so that all events are recorded properly.

This helps in debugging and monitoring the system.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s",
    filename="app.log",
    filemode="a"
)

Explanation

Here we configure logging so that all important messages are saved in a file. This allows us to track application behavior even after it stops running.

The format includes timestamp, log level, and message, which makes debugging easier. In simple terms, logging helps you understand what is happening inside your system in real-world scenarios.

3. Custom Exceptions (exceptions.py)

In real applications, we should not use generic errors for everything.

Instead, we define custom exceptions so that we can handle different error cases clearly.

This makes the system more readable and maintainable.

class APIError(Exception):
    pass

class APITimeoutError(APIError):
    pass

class APIConnectionError(APIError):
    pass

Explanation

In this code, we define custom exception classes for different types of API failures. This helps us understand exactly what kind of error occurred.

Instead of handling all errors in the same way, we can now treat timeout, connection, and API errors differently. In simple terms, this improves error handling and makes debugging easier.

4. Data Model (user_model.py)

Instead of working with raw API data, we define structured models.

This ensures that data always follows a fixed format and reduces errors.

It also improves code readability.

from dataclasses import dataclass

@dataclass
class User:
    id: int
    name: str

Explanation

In this code, we define a user model using a data class. This converts API response data into a structured object instead of a plain dictionary.

This makes the code cleaner and easier to understand. It also ensures that required fields are always present, which reduces bugs and improves reliability.

5. API Client (api_client.py)

Now we build the core component that handles API calls.

This includes retry logic, timeout handling, and connection reuse for better performance.

This is the most important part of the system.

import requests

import logging

import time

from config.config import BASE_URL, TIMEOUT, RETRIES, RETRY_DELAY

from core.exceptions import APITimeoutError, APIConnectionError, APIError



class APIClient:

    def __init__(self):

        self.session = requests.Session()



    def get(self, endpoint):

        url = f"{BASE_URL.rstrip('/')}/{endpoint.lstrip('/')}"



        for attempt in range(1, RETRIES + 1):

            try:

                logging.info(f"[Attempt {attempt}] Calling: {url}")



                response = self.session.get(url, timeout=(3, TIMEOUT))

                response.raise_for_status()



                return response.json()



            except requests.exceptions.Timeout:

                logging.warning("Timeout occurred")

                time.sleep(RETRY_DELAY * attempt)

                if attempt == RETRIES:

                    raise APITimeoutError()



            except requests.exceptions.ConnectionError:

                logging.warning("Connection issue")

                time.sleep(RETRY_DELAY * attempt)

                if attempt == RETRIES:

                    raise APIConnectionError()



            except requests.exceptions.HTTPError as e:

                logging.error(f"HTTP error: {e}")

                raise APIError(str(e))

Explanation

In this code, we are building a production-level API client. We use requests.Session() so that connections are reused, which improves performance and reduces latency.

We also implement retry logic with exponential backoff, which means delay increases after each failure. This prevents overloading the API and makes the system more stable. In simple terms, this ensures reliable and efficient API communication.

6. Response Validation (validators.py)

Before using API data, we must validate it.

This ensures that the response contains expected fields and correct format.

It helps avoid errors later in the system.

def validate_user_response(data):

    if not isinstance(data, dict):

        raise ValueError("Invalid response format")



    if "id" not in data or "name" not in data:

        raise ValueError("Missing required fields")



    return data

Explanation

In this code, we check whether the API response is valid before using it. First, we ensure the response is a dictionary.

Then we verify required fields like id and name. If something is missing, we raise an error. In simple terms, validation ensures that only correct and expected data flows into the system.

7. Service Layer (user_service.py)

Now we connect API calls with business logic.

This layer acts as a bridge between API client and application logic.

It keeps the system clean and maintainable.

from clients.api_client import APIClient

from validators.validators import validate_user_response

from models.user_model import User



class UserService:

    def __init__(self, client=None):

        self.client = client or APIClient()



    def get_user(self, user_id):

        data = self.client.get(f"users/{user_id}")

        validated = validate_user_response(data)



        return User(**validated)

Explanation

In this code, we create a service layer that uses the API client to fetch data. Instead of calling APIs directly, we use this layer to manage business logic.

We also validate the response and convert it into a structured model. In simple terms, this keeps the system organized and ensures clean data flow.

8. Main File (main.py)

import core.logger
from services.user_service import UserService

def main():
    service = UserService()

    try:
        user = service.get_user(1)
        print(user)

    except Exception as e:
        print("Error:", e)

if __name__ == "__main__":
    main()

Explanation

16. Real failure scenarios in API calls

High-Impact Failure Scenarios (Real Production Cases):

Final Takeaway

processed_requests = set()

def process_payment(request_id, amount):
    if request_id in processed_requests:
        return "Payment already processed"

    # Process payment (dummy logic)
    print(f"Charging ₹{amount}")

    processed_requests.add(request_id)
    return "Payment successful"


# First call
print(process_payment("req123", 100))

# Retry (duplicate request)
print(process_payment("req123", 100))

Explanation:

import requests
response = requests.get("https://api.example.com/payment")
print("Payment status:", response.status_code)
print("Next step executed after payment response")

Explanation:

import asyncio

async def send_email():
    print("Sending email...")
    await asyncio.sleep(2)  # simulate delay
    print("Email sent")

async def main():
    asyncio.create_task(send_email())  # runs in background
    print("Continue other work without waiting")

asyncio.run(main())

Output:

import requests
import logging
import time

logging.basicConfig(level=logging.INFO)

url = "https://api.example.com/data"

start_time = time.time()

try:
    logging.info("Calling API...")

    response = requests.get(url, timeout=5)

    response.raise_for_status()

    data = response.json()
    logging.info(f"Success | Status: {response.status_code}")

except requests.exceptions.RequestException as e:
    logging.error(f"API Error: {e}")

end_time = time.time()

latency = end_time - start_time

logging.info(f"Response Time: {latency:.2f} seconds")

Explanation:

19. Frequently Asked Questions (FAQ)

20. Summary: Python Requests Tutorial