Python

Two open source deep learning frameworks:

Caffe: http://caffe.berkeleyvision.org/

TensorFlow: https://www.tensorflow.org/ by Google

A few useful Python libraries:

requests : an HTTP client easy to use.

reportlab: A PDF library to handle text, drawings, and other graphics. Fairly comprehensive…

svglib: Convert SVG to other formats (e.g.., PDF and PNG) and embed an SVG (with scaling capability) in a PDF file assisted with reportlab. It returns a Drawing object that reportlab could handle. A small library.

cairosvg: another SVG library.

svgwrite: a library to write SVG file

Question: Converting json-like string to dict?

I have a script that changes a dict to a string and saves it to a file. I’d like to then load that file and use it as a dict, but it’s a string. Is there something like int(“7”) that can change a string formatted as a dict ({a: 1, b: 2}) into a dict? I’ve tried dict(), but this doesn’t seem to be what it does. I’ve heard of some process involving JSON and eval(), but I don’t really see what this does. The program loads the same data it saves, and if someone edits it and it doesn’t work, it’s no problem of mine (I don’t need any advanced method of confirming the dict data or anything).

Answer:

Try this, it’s the safest way:

import ast
ast.literal_eval("{'x':1, 'y':2}")
=> {'y': 2, 'x': 1}

All the solutions based in eval() are dangerous, malicious code could be injected inside the string and get executed.

According to the documentation the expression gets evaluated safely. Also, according to the source code, literal_eval parses the string to a python AST (source tree), and returns only if it is a literal. The code is never executed, only parsed, so there is no reason for it to be a security risk.

Question: How to make “python” command execute python3

Answer: A simple safe way would be to use an alias.

Place this into ~/.bashrc or ~/.bash_aliases file:

alias python=python3

(alias python=”/home/whoever/anaconda3/bin/python”)

After adding the above in the file, run the command below:

source ~/.bash_aliases or source ~/.bashrc

For example:

$ python --version
Python 2.7.6
$ python3 --version
Python 3.4.3
$ alias python=python3
$ python --version
Python 3.4.3

To circumvent the alias use the command built-in command:

$ command python --version
Python 2.7.6

Another way to circumvent the alias is to use \ before the command.

$ \python --version 
Python 2.7.6

To disable the alias in the current shell use the unalias built-in command:

$ unalias python
$ python --version
Python 2.7.6

It is not recommended to change the symbolic link (/usr/bin/python) because of other package dependencies.

Multi-threading in PythonCPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing or concurrent.futures.ProcessPoolExecutor. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously. Personal note: these features are very similar to multi-threading in Java (such as executor model). You may use submit() or map(), depending on your need. Map corresponds to an iterable (such as a list, dict, set, etc.).

HTTP client library: requests, requests-oauthlib, and requests-toolbelt (multipart/form-date encoder).

To include a parameter corresponding to a JSON string in a HTTP request, try json.dumps(). This will convert the input to a JSON string. For example,

payload = { 'some key': json.dumps(parameter) }
response = request.post(url, params=payload, auth=auth)
How to use python virtualenv with a particular version of python (e.g., not through anaconda) on Linux

By default, pip installs in /usr/local, which needs sudo permissions. Instead, whenever you are working with Python modules, use virtualenv. It’s pretty easy:

  1. Globally install Python virtualenv 
    sudo pip install virtualenv
  2. Create a virtualenv 
    virtualenv --python=/usr/local/bin/python3.8 myenv
  3. Activate your virtualenv 
    source myenv/bin/activate
  4. Install Django locally in the virtualenv 
    pip install django

This way you don’t have to worry about modifying global Python modules and its much safer.

How to package python application into a stand-alone executable for various platforms?

PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app without installing a Python interpreter or any modules. PyInstaller supports Python 2.7 and Python 3.5+, and correctly bundles the major Python packages such as numpy, PyQt, Django, wxPython, and others.

How to install Python from the source on Ubuntu?

Step#1 Update the packages list and install the packages necessary to build Python:

sudo apt update
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget

Step#2 Download the source code from the official Python repository

wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz

Step#3 Extract the downloaded file

tar -xf Python-3.8.2.tgz

Step#4 Switch to the folder and execute the configure script which performs a number of checks to make sure all of the dependencies on your system are present:

cd Python-3.8.2
./configure --enable-optimizations

Step#5 Start the build process

make -j 4 (assume you are going to use 4 cores to speed up the build process)

Step#6 Install the binaries

sudo make altinstall

How to install a different version of Python while using conda?

First, you will need to create a new environment (e.g., myenv)

conda create --name myenv

Then, activate the new environment

conda activate myenv

Third, you will first search to verify the available python packages.

conda search python

Fourth, you could now install the new version of python

conda install python=3.8.1

Asyncio module in Python

With python 3.5 you can use the new await/async syntax:

import asyncio
import requests

async def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = await future1
    response2 = await future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())