Python

Two open source deep learning frameworks:

Caffe: http://caffe.berkeleyvision.org/

TensorFlow: https://www.tensorflow.org/ by Google

A few useful Python libraries:

requests : an HTTP client easy to use.

reportlab: A PDF library to handle text, drawings, and other graphics. Fairly comprehensive…

svglib: Convert SVG to other formats (e.g.., PDF and PNG) and embed an SVG (with scaling capability) in a PDF file assisted with reportlab. It returns a Drawing object that reportlab could handle. A small library.

cairosvg: another SVG library.

Question: Converting json-like string to dict?

I have a script that changes a dict to a string and saves it to a file. I’d like to then load that file and use it as a dict, but it’s a string. Is there something like int(“7”) that can change a string formatted as a dict ({a: 1, b: 2}) into a dict? I’ve tried dict(), but this doesn’t seem to be what it does. I’ve heard of some process involving JSON and eval(), but I don’t really see what this does. The program loads the same data it saves, and if someone edits it and it doesn’t work, it’s no problem of mine (I don’t need any advanced method of confirming the dict data or anything).

Answer:

Try this, it’s the safest way:

import ast
ast.literal_eval("{'x':1, 'y':2}")
=> {'y': 2, 'x': 1}

All the solutions based in eval() are dangerous, malicious code could be injected inside the string and get executed.

According to the documentation the expression gets evaluated safely. Also, according to the source code, literal_eval parses the string to a python AST (source tree), and returns only if it is a literal. The code is never executed, only parsed, so there is no reason for it to be a security risk.

Question: How to make “python” command execute python3

Answer: A simple safe way would be to use an alias.

Place this into ~/.bashrc or ~/.bash_aliases file:

alias python=python3

(alias python=”/home/whoever/anaconda3/bin/python”)

After adding the above in the file, run the command below:

source ~/.bash_aliases or source ~/.bashrc

For example:

$ python --version
Python 2.7.6
$ python3 --version
Python 3.4.3
$ alias python=python3
$ python --version
Python 3.4.3

To circumvent the alias use the command built-in command:

$ command python --version
Python 2.7.6

Another way to circumvent the alias is to use \ before the command.

$ \python --version 
Python 2.7.6

To disable the alias in the current shell use the unalias built-in command:

$ unalias python
$ python --version
Python 2.7.6

It is not recommended to change the symbolic link (/usr/bin/python) because of other package dependencies.

Multi-threading in Python

CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing or concurrent.futures.ProcessPoolExecutor. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously. Personal note: these features are very similar to multi-threading in Java (such as executor model). You may use submit() or map(), depending on your need. Map corresponds to an iterable (such as a list, dict, etc.).

HTTP client library: requests, requests-oauthlib, and requests-toolbelt (multipart/form-date encoder).

To include a parameter corresponding to a JSON string in a HTTP request, try json.dumps(). This will convert the input to a JSON string. For example,

payload = { 'some key': json.dumps(parameter) }
response = request.post(url, params=payload, auth=auth)

How to use python virtualenv when using default python (e.g., not through anaconda) on Linux

By default, pip installs in /usr/local, which needs sudo permissions. Instead, whenever you are working with Python modules, use virtualenv. It’s pretty easy:

  1. Globally install Python virtualenv
    sudo pip install virtualenv
    
  2. Create a virtualenv
    virtualenv myenv
    
  3. Activate your virtualenv
    source myenv/bin/activate
    
  4. Install Django locally in the virtualenv
    pip install django
    

This way you don’t have to worry about modifying global Python modules and its much safer.

How to package python application into a stand-alone executable for various platforms?

PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app without installing a Python interpreter or any modules. PyInstaller supports Python 2.7 and Python 3.5+, and correctly bundles the major Python packages such as numpy, PyQt, Django, wxPython, and others.