Proxy locations

Europe

North America

South America

Asia

Africa

Oceania

See all locations

Network statusCareers

Back to blog

HTTPX vs Requests vs AIOHTTP

Danielius Radavicius

2023-02-205 min read
Share

HTTPX is a modern HTTP client for Python that aims to provide a more enjoyable and powerful experience for making HTTP requests. It was created to add support for asynchronous programming and has quickly gained popularity in the Python community due to its feature-rich API and good performance. In this article, we will compare HTTPX to two other popular HTTP clients for Python: Requests and AIOHTTP.

How HTTPX compares to the Requests module

Requests is a widely-used Python library, especially for making HTTP requests. Its simple API and long existence make it the go-to choice for many developers. To learn more about Requests, read our comprehensive guide to Python Requests Library.

HTTPX is a newer and more feature-rich HTTP client for Python than the popular Requests library. Here are some of the key differences between HTTPX and Requests:

  • Async compatibility: HTTPX has both async and sync compatibility, which means you can use it in both asynchronous and synchronous programs. The Requests library, on the other hand, is only synchronous.

  • HTTP/2 support: HTTPX supports HTTP/2, the latest HTTP protocol version. The Requests library does not have this feature.

  • Automatic decoding: HTTPX includes automatic decoding of JSON and other common formats, which can save developers time and effort when working with responses that contain structured data

  • Size: HTTPX is larger than Requests, which may be a consideration for applications where size is a concern.

  • Performance: HTTPX has better performance compared to Requests in most cases.

Below are syntax differences between the two. The following code uses the Requests library to send a GET request:

import requests
response = requests.get("https://example.com")
print(response.text)

The equivalent code uses the HTTPX library:

import httpx
response = httpx.get("https://example.com")
print(response.text)

Alternatively, you can create an httpx client object and use its get method:

client = httpx.Client()
response = httpx.get("https://example.com")

If you want to send a POST request using the Requests library, use the post method instead of get as follows:

response = requests.post(url, data= {"name": "John", "age": 30} )

The syntax using HTTPX is very similar:

client = httpx.Client()
response = client.post("https://httpbin.org/post",data={"name": "John", "age": 30} )

The two libraries are used almost identically in simple synchronous situations. The important difference is that HTTPX allows you to write asynchronous code as follows:

import httpx
import asyncio

async def main():
    async with httpx.AsyncClient() as client:
        response = await client.get("https://example.com")
        print(response.text)

asyncio.run(main())

As you can see, the syntax for making a request with HTTPX is slightly more verbose due to Python's async/await syntax. However, this is a small price to pay for the ability to make asynchronous HTTP requests.

HTTPX also includes automatic decoding of JSON and other common formats, which can save developers time and effort when working with responses that include structured data.

See the following example:

import httpx
response = httpx.get("http://httpbin.org/get")
data = response.json()
print(data["headers"]["User-Agent"])

The site http://httpbin.org/get returns a JSON response.

Do note that there is a new version of the HTTP protocol called HTTP/2, which provides far more efficient transport and better performance. If you want to read more about HTTP/2, see this detailed document. Importantly, HTTPX offers support for HTTP/2 while Requests doesn’t.

To utilize HTTP/2 with the HTTPX library, first, install the optional HTTP/2 component using the pip command:

$ pip install 'httpx[http2]'

Next, set the optional http2 flag to True when you create an instance of the client as follows:

client = httpx.Client(http2=True)
response = client.get("https://example.com")
print(response.http_version) # Prints HTTP/2

This snippet prints the currently used HTTP protocol version, which in this case is HTTP/2.

Features that make HTTPX stand out 

Several features make HTTPX stand out as a powerful and flexible HTTP client for Python. These features include support for async/await syntax, sync compatibility, HTTP/2 protocol, and support from streaming response.

If you need to download large amounts of data, you may want to use streaming responses that don't load the entire response body into memory at once. Thankfully, HTTPX has built-in support for streaming responses.

It’s worth noting that the similarly popular library AIOHTTP, offers a plethora of valuable options as well.

An overview of AIOHTTP

AIOHTTP is a client for Python that is designed for use in asynchronous programs. Similarly to HTTPX, it’s built on top of the asyncio library and supports async/await syntax meaning AIOHTTP is a good choice for developers who are already familiar with asyncio and want a lightweight library for making HTTP requests. To find out more, visit the official website.

Like HTTPX, AIOHTTP supports standard HTTP methods. It also includes support for cookies, redirects, and custom headers. AIOHTTP is particularly well-suited for making HTTP requests in high-concurrency environments due to its async-first design and efficient use of resources.

Here is an example of making an asynchronous GET request using AIOHTTP:

import aiohttp
import asyncio

async def main(): 
    async with aiohttp.ClientSession() as session: 
            async with session.get("https://example.com") as response: 
                print(await response.text())

asyncio.run(main())

AIOHTTP also supports making asynchronous POST requests and other HTTP methods similarly. Here is an example of making a POST request with AIOHTTP:

import aiohttp
import asyncio

async def main(): 
    async with aiohttp.ClientSession() as session: 
            async with session.post("http://httpbin.org/post", data={"key": "value"}) as response: 
                print(await response.text())

asyncio.run(main())

If you want to learn more about AIOHTTP, we highly recommend reading Asynchronous Web Scraping with AIOHTTP.

HTTPX vs AIOHTTP

When comparing HTTPX and AIOHTTP, one of the main differences is the scope of their libraries. HTTPX is a full-featured HTTP client that can be used for a wide range of applications, while AIOHTTP is more focused on providing a simple and efficient API primarily for making HTTP requests in async programs.

Another difference is that HTTPX supports synchronous and asynchronous programming, while AIOHTTP is strictly an async library. Furthermore, AIOHTTP doesn’t merely send HTTP requests but also allows the creation of HTTP servers.

Usually, HTTPX can be used in a broader range of projects, while AIOHTTP may be a better choice if you are working on async-only projects and want a library optimized for async programming.

Overall, HTTPX and AIOHTTP are powerful and flexible HTTP clients for Python that have a multitude of features and are well-suited for different projects. However, there are a few key differences between the two libraries:

  • Async compatibility: HTTPX has both async and sync compatibility, while AIOHTTP is async-only.

  • HTTP/2 support: HTTPX supports HTTP/2, the latest HTTP protocol version. AIOHTTP does not have this feature.

  • Performance: AIOHTTP is generally considered to have excellent performance compared to HTTPX. See the next section for a detailed analysis.

Notably, the choice between the two will depend on your specific needs and requirements.

Performance Comparison of AIOHTTP and HTTPX

To compare the performance of AIOHTTP and HTTPX, we can build a simple program that sends multiple asynchronous GET requests to a website. Here is an example of such a program:

import asyncio
import time
import httpx
import aiohttp

async def main():
    # Create clients for both the library
    httpx_client = httpx.AsyncClient()
    aiohttp_client = aiohttp.ClientSession()

    try:
        # Send 100 asynchronous GET requests using HTTPX
        start_time = time.perf_counter()
        tasks = [httpx_client.get("https://example.com") for _ in range(100)]
        await asyncio.gather(*tasks)
        end_time = time.perf_counter()
        print(f"HTTPX: {end_time - start_time:.2f} seconds")

        # Send 100 asynchronous GET requests using AIOHTTP
        start_time = time.perf_counter()
        tasks = [aiohttp_client.get("https://example.com") for _ in range(100)]
        await asyncio.gather(*tasks)
        end_time = time.perf_counter()
        print(f"AIOHTTP: {end_time - start_time:.2f} seconds")
    finally:
        # Close client sessions
        await aiohttp_client.close()
        await httpx_client.aclose()

asyncio.run(main())

Here are the results of running this code on a mac with an M1 processor and gigabit internet:

HTTPX: 1.22 seconds

AIOHTTP: 1.19 seconds

If we run the same code for 1000 requests, the numbers change significantly:

HTTPX: 10.22 seconds

AIOHTTP: 3.79 seconds

As we can see, AIOHTTP is faster than HTTPX in this case. However, it’s important to note that the performance difference between the two libraries may vary depending on the specific use case and hardware.

Comparison of HTTPX vs Requests vs AIOHTTP

The following table shows a summary of comparison of HTTPX vs Requests vs AIOHTTP :

Conclusion

Ultimately, the choice of an HTTP client library will depend on the specific needs of your project. Requests may be the best choice if you need a simple and easy-to-use library. If you need a more feature-rich library with async support, HTTPX could be a better fit.

It's worth noting that HTTPX and AIOHTTP are not the only options for making HTTP requests in Python. Many other libraries are available, each with its own features and trade-offs. Some other popular choices include http.client, urllib3, and httplib2.

Also, if you would like to try an all-in-one web scraping solution, check our Web Scraper API to gather real-time public information from most websites.

About the author

Danielius Radavicius

Copywriter

Danielius Radavičius is a Copywriter at Oxylabs. Having grown up in films, music, and books and having a keen interest in the defense industry, he decided to move his career toward tech-related subjects and quickly became interested in all things technology. In his free time, you'll probably find Danielius watching films, listening to music, and planning world domination.

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.

Related articles

Get the latest news from data gathering world

I’m interested

IN THIS ARTICLE:


  • How HTTPX compares to the Requests module

  • Features that make HTTPX stand out 

  • An overview of AIOHTTP

  • HTTPX vs AIOHTTP

  • Performance Comparison of AIOHTTP and HTTPX

  • Comparison of HTTPX vs Requests vs AIOHTTP

  • Conclusion

Forget about complex web scraping processes

Choose Oxylabs' advanced web intelligence collection solutions to gather real-time public data hassle-free.

Feature HTTPX Requests AIOHTTP
Async compatible Yes No Yes
Sync compatible Yes Yes No
Automatic JSON decoding Yes Yes No
HTTP/2 support Yes No No
Cookies Yes Yes Yes
Redirects Yes Yes Yes
Authentication Yes Yes Yes
Custom headers Yes Yes Yes
Streaming responses No No No
Size Large Smaller Smaller
Performance Good Good Excellent