Posts

Python: Consuming HTTP APIs

Hyper Text Transfer Protocol (HTTP) based APIs are ubiquious in the Internet these days. Whether they are in the form of Simple Object Access Protocol (SOAP), Representational State Transfer (REST) or simple GET/POST requests, HTTP based API's are the de-facto standard for consuming services over the web. In this article, we take a look how to consume these HTTP API's from Python. There are three main modules that can be used to make HTTP API calls in Python, urllib , httplib and requests . Under the hood, requests uses urllib which in turn uses some functionality of httplib . Hence requests provides the most abstraction and ease of use among the three and we will be using that in this article. The following code shows how to make simple GET and POST requests using the requests module. The GET will download the main page of the site https://cbelwal.blogspot.com which the POST will make a POST call and will get the 405 error. Note that we are accessing the URL over SSL an

Python: JSON Serialization / DeSerialization

JaveScript Object Notation (JSON) has become the de-facto convention when passing data through Internet web services, primarily those using the Representational State Transfer (REST) APIs. Originally developed for JavaScript JSON is a simple, human readable format of key-value pairs that is now supported by all major languages and platforms. The following is a simple JSON containing the Name, Age and City of a person. {"name": "John Doe", "age": 35, "country": "US"} A popular way of transferring data is to serialize/de-serialize data objects using JSON functionality supported in the platform. For example, if we have the following class in Python: class dataClass: def __init__(self): self.name = "John Doe" self.age = 35 self.country = "US" The serialized version of this class in JSON is the one we gave earlier,: {"name": "John Doe", "age": 35,

Python: Sockets Programming - Multi-Threaded Server

Image
In a previous article , we built a simple socket server that uses only one thread to service client connections, allowing it to service only one client at a time. New clients were able to connect but could not receive or send data to/from the socket server. In this article, we look at how to create a multi-threaded socket server to serve an unlimited number of clients. We use the same code base as the single threaded server and extend it to create multiple threads. The parent socket sets up a listening socket as before and when new clients connect a new child socket object is created. Unlike the single threaded server, we pass the child socket object to a new thread class which then handles all communication with the client in its own thread. Using a while loop the parent socket then goes back to listen for more connections and creates new threads for new connections passing the reference of the child sockets which are created.The code that executes in it own thread and communica

Python: Sockets Programming - Single Thread Server

Image
In previous articles , we built a socket client and made a connection to an Internet based web server. The Internet web server was a socket server listening on port 80. In this article, we discuss how to create a TCP socket server in Python and make it listen to a port we specify. We can then use any socket client, including the one we created in previous chapters, to communicate with the socket server. Note that Python has a dedicated module SocketServer to create socket servers which internally uses the socket module. In this article we do not use the SocketServer module and show how to build a server using the socket module. The code to run for a socket server is slightly different that than of s socket client. We first have to create a socket object to bind (using socket.bind() ) to a specific hostname and port (a hostname/port combination is also called a address ) and setup the parent socket to accept inbound connections from clients. When a client connects to the specifi

Python: Sockets Programming - Non-blocking Client

Image
Sockets are an operation system construct allowing Inter-Process Communication (IPC) over a network. Sockets is how you are able to browse the internet, watch movies or download files from your computer. In a previous article we built a sockets client using blocking calls. In this article, we modify the code of the blocking socket client to make the data receive and send non-blocking. The socket mode for blocking/non-blocking can be set using the setblocking(value) or settimeout(value) functions of the socket class. Another important function is select which is a blocking/non-blocking (depending on mode) and is used to determine if the socket has data to read. Without select if we try to read data and there is none, we will get an exception which will have to be handled. Using select is a more cleaner way (compared to a try-except block) of doing non-blocking calls so that exceptions are used only for capturing genuine errors. We also need a while loop to execute any non

Python: Sockets Programming - Blocking Client

Image
Sockets are an operation system construct allowing Inter-Process Communication (IPC) over a network. Sockets is how you are able to browse the internet, watch movies or download files from your computer. The socket module in Python allows us to create a socket connection to a remote server or create a listening socket acting as a server. Under the hood, the socket module uses operation system calls to communicate with sockets, hence there are some small differences in using sockets in Windows and UNIX/Linux based systems. In Python, socket calls to send or receive data can be blocking or non-blocking. Blocking calls wait and do not go to the next line of execution till the required data is full or the socket is closed. Non-blocking calls are asynchronous operations that can execute the next line of code even if data is coming as they call a separate function to handle the sending/receiving of data. In this article the client we only uses blocking socket calls, and we have show

Python: Multi-threaded programming

Multi-thread programming is an important component of a programming platform which allows specific functions in the code to execute without blocking, also known as asynchronous execution. Threads are also used for parallel, or near to parallel (in case threads execute in the same core/CPU) execution of a function in the code. Threads essentially solve two problems: 1. Make the application and user interface responsive by executing I/O tasks in a separate thread 2. Make the application and user interface responsive by execution CPU intensive algorithms in a separate thread In Python, multi-threading is only used in #1, since #2 is not possible due a mutex that protects all Python objects (a.k.a. Global Interpreter Lock or GIL). GIL is necessary since memory management in Python is not thread safe, and now so many features of Python depend on GIL that it is no easy task to remove it. There is some good news. GIL is only used in the CPython implementation of Python and the other