Requests Module or Request Library in Python

In this part of Learning Python we Cover Requests module in python
Written by Paayi Tech |10-May-2019 | 0 Comments | 68 Views

In this section, we will see how to send actual requests and fetch the responses. For that purpose, we are taking the help of a website that is as follows:

https://reqres.in/

Now the question arises what this website does? This website provides the dummy data in rest API format. This website is designed for practicing the requests methods in python or ajax in case of javascript programming. This website provides all type of requests methods. That gets, post, put delete, etc. Here we will see what sort of responses came back and what happened if the requests are not right. So, let’s start with the get requests:

 

Get Request:

As we know the get requests are used to fetch the data so let’s see the code:

import requests
r = requests.get('https://reqres.in/api/users?page=2')
r = r.json()
print(r)

 

The above code gets the data from the URL. After getting the data, we convert it into JSON format and then print the request. Following will be the output generated as a result:

{

    "page": 2,

    "per_page": 3,

    "total": 12,

    "total_pages": 4,

    "data": [

        {

            "id": 4,

            "first_name": "Eve",

            "last_name": "Holt",

            "avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/marcoramires/128.jpg"

        },

        {

            "id": 5,

            "first_name": "Charles",

            "last_name": "Morris",

            "avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/stephenmoon/128.jpg"

        },

        {

            "id": 6,

            "first_name": "Tracey",

            "last_name": "Ramos",

            "avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/bigmancho/128.jpg"

        }

    ]

}

 

As we can see that we have given the full URL in which the parameters are also attached. In the above case page is a parameter whose value is 2. We can convert the above code as following which makes it easier to understand, and the code will also become easy to know when there are lots of parameters:

import requests
payload = {'page':2}
r = requests.get('https://reqres.in/api/users', params=payload)
r = r.json()
print(r)

 

Now we will what is the status code is generated as a result of a request so if we print r.status_code. The following output will be given:

import requests
payload = {'page':2}
r = requests.get('https://reqres.in/api/users', params=payload)
 
print(r.status_code)

Output

200

 

200 status code indicates that a request is successful.

 

Error in Response:

Sometimes there could be an error which in return gives the negative response. Following is the example of an error response. The requests that we are going to make is not present that is why the response that is generated is empty, and the status code will be of 404

Following is the code:

import requests
 
r = requests.get('https://reqres.in/api/users/23')
print("Error: " + str(r.status_code))
 
print('Response: ' + str(r.json()))

 

The output of the above-written code is as follows:

Error: 404

Response: {}

 

This error is generated because no such user is presented in the database which results in the error in the query.

This error can be avoided by using the try-except statement. However, this can only be done during production so that no one can see the error by which anyone can manipulate with the code, only during debugging so that it is easy to debug our code.

 

Post Requests:

Now we will see how to handle post requests. Post requests are used when we have to send the data confidentially or to add the data. The difference between post data and get data is, in getting data anyone can see the information that is being sent, but in the post, requests data is hidden and can’t be seen. Following is the example of post requests:

import requests
 
payload = {"name": "Hassan Rehman", "job":"Data Scientist"}
r = requests.post('https://reqres.in/api/users', params=payload)
print("Status: " + str(r.status_code))
 
print('Response: ' + str(r.json()))

 

In the above example, we set a payload in which name and job are assigned in the dictionary. Then we post data to create a new user, and along with this, we have sent the payload that we assigned earlier. The output of the above code is as follows:

Status: 201

Response: {'id': '817', 'createdAt': '2019-02-12T18:35:08.769Z'}

 

The output shows the status code that is 201 which indicates successfully created new user and response shows the id at which new user is created and the time of creation.

Request Delete:

The delete request as the name indicates to delete the request and If we print the response, it will generate an error because there is no data left to show as a response. Following is the example of delete request in python:

import requests
 
 
r = requests.delete('https://reqres.in/api/users/2')
print("Status: " + str(r.status_code))
 
print('Response: ' + str(r.json()))

 

When we print the status, it will be 204 which indicates the data has been deleted, but when we see the response, it will show an error because no data to display. The output is as follows:

Status: 204

 

return self.scan_once(s, idx=_w(s, idx).end())

simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

 

The JSON was expecting a value, but none was found.

 

Put Request:

Put request is used to edit the data. To verify this, we try to update the data and in response, the time of updating will be set. The code is as follows:

 

import requests
 
payload = {"name":"Hassan Rehman", "job": "Data Scientis"}
r = requests.put('https://reqres.in/api/users/2', params = payload)
print("Status: " + str(r.status_code))
 
print('Response: ' + str(r.json()))

 

The output will show the updated time that indicated it is updated:

Status: 200

Response: {'updatedAt': '2019-02-12T18:53:50.481Z'}

 

Getting HTML response:

As we studied earlier that getting the HTML response is used when we try to scrape the data. The response gives the HTML, and then we parse the data and extract the information we needed. Following is the code to fetch the HTML from the website:

import requests
 
 
r = requests.get('https://www.python.org/')
print(r.text)

 

If we print the r.text it will generate the HTML as follows:

Figure 1

 

The above image is the print statement. It contains all the page source that is needed to scrape we can get the data from this.

It is all the request module. In the next section, we will learn web scraping by using this requests module to scrape the python.org website and what are the uses of scraping the data and in which department it is used extensively. We will also learn about the beautiful soup in the next section.



Login/Sign Up

Comments




Related Posts



© Copyright 2019, All Rights Reserved. paayi.com

This site uses cookies. By continuing to use this site or clicking "I Agree", you agree to the use of cookies. Read our cookies policy and privacy statement for more information.