IDEs and Tools

Python IDEs and Debuggers

Picking a IDE with a powerful graphic debugger is CRITICAL in program development!!!

Python's Anaconda Distribution (Recommended for Data Science)

Besides the Official Python Distribution from, there are many Python distributions available. Among them, the open-source Anaconda Python Distribution (@ is probably the most popular for data science, by bundling:

  • IDEs: Jupyter Notebook, Spyder
  • Analytics & Scientific Computing Python Packages: NumPy, Matplotlib, Pandas, SciPy
  • Machine Learning Python Packages: scikit-learn

It is available on Linux, Windows, and Mac OS X.


Goto Anaconda's mother site @ Download "Anaconda Distribution". Choose the Python 3 version. Follow the instructions to install.

Getting Started

The Anaconda Distribution comes with:

  • Anaconda Prompt: a Command-line shell (CMD, Terminal, Bash Shell) for launching the Python Interactive Interpreter.
    (base) xxx> python
    Python 3.7.0 (default) [MSC v.1912 64 bit (AMD64)] :: Anaconda custom (64-bit) on win32
    Type "help", "copyright", "credits" or "license" for more information.
  • Jupyter Notebook: a web-based IDE suitable for data analytics by combining code, documentation, and visual into notebook.
  • Spyder: a MatLab-like IDE.
  • Anaconda Navigator

Explore "Get Started" section, under

Conda Package Management System

Anaconda uses the open-source conda package management system for managing packages and their dependencies (install and update), and the environment.

You need to run Conda under the "Anaconda Prompt". To install or update pacakges, you may need the administrator/superuser right:

  • For Windows, right-click and "run as administrator"
  • For Linux and Mac OS X, run the commands with "sudo".

Using Conada:

  1. Conda's commands:
    # Show help menu
    $ conda help
    # Show help for command
    $ conda command --help
    # Clean up the conda installation - remove cache, outdated packages, zip files
    $ conda clean --all
    # Show info about Anaconda
    $ conda info
    # Show all packages
    $ conda list
    # Install a package
    $ conda install package-name
    # Update a package
    $ conda update package-name
  2. To install a package under Anaconda, start a Anaconda Prompt (as administrator or with sudo) and issue:
    $ conda install --help
    $ conda install package_name
    # For examples,
    $ conda install numpy
    $ conda install pandas
  3. To update a package, start a Anaconda Prompt (as administrator or with sudo) and issue:
    $ conda update --help
    # Update Anaconda distribution
    $ conda update anaconda
    # Update a Python package
    $ conda update numpy
    $ conda update pandas
    $ conda update matplotlib
    $ conda update scikit-learn
    $ conda update jupyter
  4. (For Windows) You can change the start-up directory of "Anaconda Prompt", by right-right on the program ⇒ Properties ⇒ Shortcut ⇒ change the "start-in".

Jupyter Notebook (Recommended for Data Science Projects)

The Jupyter Notebook (@ is an open-source web application that allows you to create and share notebooks. According to Wiki, "A notebook is a book or binder of paper pages, often ruled, used for purposes such as recording notes or memoranda, writing, drawing or scrapbooking." A Jupyter Notebook (just like a paper notebook) integrates code and its output into a single document that combines visualizations, narrative text, mathematical equations, and other rich media.

Jupyter Notebook is an easy-to-use data science tool for beginners and educators, in particular for data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning. It supports many programming languages, of which Python is the most commonly-used.

Jupyter Notebook is a successor to the earlier iPython Notebook, started in 2010.


Follow the instructions at the Jupyter mother site (@ Download and install "Anaconda" (Python 3 version), which is the most widely used Python distribution for data science, bundled with Jupyter Notebook and Python data science libraries such as NumPy, pandas and Matplotlib.

Getting Started

References: Read "Jupyter Notebook for Beginners: A Tutorial" @, which has a simple data science example.

  1. Launch "Jupyter Notebook", which is a webapp that opens a new tab in your default web browser, showing the so-called "Jupyter Dashboard", with URL http://localhost:8888/tree listing the default start-up directory.
  2. The default start-up directory is probably not your desired working directory. To change the start-up directory, shutdown the previous app (Press Ctrl-C twice at the console). Launch the Anaconda Prompt from the Start menu (Windows). Change directory to your desired start-up directory and run "jupyter notebook".
  3. To create a new notebook for Python, click "New" and select "Python". A new notebook is opened in a new tab. It is saved as "Untitled.ipynb" (switch back to the Dashboard to see this new file in the directory). Each .ipynb file is a text file that describes the contents of your notebook in JSON format. You can check its contents later after performing some operations.
  4. Jupyter's code is organized in fragments called cells. You can create/remove cells and re-arrange them to create your desired notebook.
  5. Type "print('Hello World!')" into the active cell labeled In[ ], and click "run" (or press Ctrl+Enter). The label changes from In[] to In[1]. The number 1 indicates when the cell was executed on the kernel. Run the cell again, it will change to In[2]. Take note of the printed output below the code.
  6. Create a new cell by choosing menu "Insert" ⇒ "Insert cell below", or click the '+' icon. Enter "print(2 ** 88)" and Run.
  7. Create a new new cell. Enter the following code. Run and take note of the "Out[n]" box, which contains the return value of the code.
    def say_hello(who):
        return 'Hello, {}!'.format(who)
  8. Create a new cell. Enter the following code.
    import time
    Run and observe that In[ ] changes to In[*] (indicating the cell is currently running the sleep()), and then In[n] when completed.
  9. To run a Python script, enter either:
    1. %run -i script_name (without the .py)
    2. exec(open('').read())
  10. Jupyter runs in two modes: edit mode (with green left-border for the active cell and a "pen" icon displayed on top-right corner) and command mode (with blue left-border). You can switch between these 2 modes by clicking inside/outside the active cell.
  11. In "command mode", you can issue these commands (with their respective keyboard shortcuts):
    • Keys 'a' and 'b' to insert a cell above and below the active cell.
    • Keys 'd'+'d' ('d' twice) to delete the active cell, and 'z' to undo the delete.
    • Shift+Up/Down (or Shift+Click the margin) to select multiple cells, and Shift+'m' to merge the cells.
  12. In "edit mode", you can issue these commands (with their respective keyboard shortcuts):
    • Ctrl+Shift+'-' to split the active cell at the cursor.
  13. To rename a notebook, you need to go to the Dashboard. Select the notebook, choose "Shutdown" to shutdown the kernel. Select the notebook again, then "Rename".
  14. To shutdown Jupyter Notebook, press Ctrl-C twice at the console.

A cell contains program code or documentation text. Jupyter Notebook supports two types of cells:

  1. Code Cell: contains program code to be executed in the kernel. The output is displayed below the cell.
  2. Markdown Cell: contains text formatted using Markdown (a lightweight version of HTML - to be explained below). The output is displayed in-place when it is run.

A kernel is a computational engine that executes the code contained in a notebook. Each notebook runs a kernel, which maintains the state of a notebook's computations. When you run a code cell, that code is run inside the kernel and the output is returned back to the cell to be displayed. The kernel's state persists over time and between cells. A variable defined or a moduled imported in one cell is available to another cell. Take note that cells are ordered which corresponds to the state of the kernel. From the "Kernel" menu, you can:

  • Restart: restart the kernel. Clearing all the variables.
  • Restart & Clear Output: Also clear the output displayed below the code cells.
  • Restart & Run All: Also run all your codes from top to bottom.
  • Interrput: If your kernel is stuck on a computation and you wish to stop it.
  • Shutdown:

Jupyter Notebook support many types of kernel. Beside the various version of Python, it supports over 100 languages including Java, C, R, Julia, MatLab, and SoS multi-language kernel.

Formatting Text with Markdown

Markdown cells are meant for including formatted documentation for your notebook. Markdown is a lightweight (or simplified or subset) of HTML, with very few markup tags for formatting documents, e.g., line beginning with # for <h1>, ## for <h2>, **text** for <strong>, __text__ for <em>, etc.

For example, insert a new cell, select "Markdown" from the pull-down menu, and enter the following:

# This is a level 1 heading or h1 in HTML
## This is a level 2 heading or h2 in HTML
### This is a level 3 heading or h3 in HTML
This is a paragraph or p in HTML.
Mark text as strong via **strong** and __strong__, or em via *emphasis* and _emphasis_.

Separate paragraphs by an empty line.

Below is an unordered list (marked by a beginning star):
* level 1
 * level 2 (indented)

Below is an order list (marked by numbers):
1. Item 1
2. Item 2

Markdown cell supports normal HTML as well.

<h1>Level 1 Heading</h1>
<p>A paragraph with <strong>strong</strong> and <em>emphasis</em>.</p>

To show the actual layout, run the cell or press Ctrl-Enter.

Take note that the Markdown cells support the normal HTML as well, which many people are more familiar and comfortable.


HARD!!! Use Spyder or IDLE instead, which supports breakpoint and stepping.


Spyder (Scientific Python Development Environment) (@ is an IDE that has similar features to the IDE of Matlab.

"Spyder is a powerful scientific environment written in Python, for Python, and designed by and for scientists, engineers and data analysts. It features a unique combination of the advanced editing, analysis, debugging, and profiling functionality of a comprehensive development tool with the data exploration, interactive execution, deep inspection, and beautiful visualization capabilities of a scientific package. Furthermore, Spyder offers built-in integration with many popular scientific packages, including NumPy, SciPy, Pandas, IPython, QtConsole, Matplotlib, SymPy, and more."

Spyder is bundled in Python's Anaconda distribution, can be installed together with Jupyter Notebook.

Getting Started

Spyder IDE contains an Editor and a IPython Console. You can use the editor to edit and run a Python script, or use the IPython Console to issue a Python statement. The editor support syntax highlighting, content-assit, and many productivity features. You can call out the online help.


The editor supports debugging. You can set breakpoints by double-clicking on the left margin, step through or run to the statements, and watch the variables and outputs.

IDLE (Recommended for Beginners)

Python IDLE (Interactive DeveLopment Environment) is a simple IDE with features such as syntax highlighting, automatic code indentation and visual debugger.

Installing/Launching IDLE
  • For Ubuntu: To install Python IDLE for Python 2 and Python 3, respectively:
    # Install IDLE for Python 2
    $ sudo apt-get install idle
    # Install IDLE for Python 3
    $ sudo apt-get install idle3
    # Verify the installation
    $ which idle
    $ which idle3
    $ ll /usr/bin/idle*
    -rwxr-xr-x 1 root root 91 xxx xx  xxxx /usr/bin/idle*
    -rwxr-xr-x 1 root root 92 xxx xx  xxxx /usr/bin/idle3*
    -rwxr-xr-x 1 root root 94 xxx xx  xxxx /usr/bin/idle-python2.7*
    -rwxr-xr-x 1 root root 94 xxx xx  xxxx /usr/bin/idle-python3.5*
    $ dpkg --status idle
    Version: 2.7.11-1
    $ dpkg --status idle3
    Version: 3.5.1-3
    To launch IDLE for Python 2 or Python 3:
    $ idle
    $ idle3
  • For Windows: IDLE is bundled in the installation. Click the START button ⇒ Python ⇒ IDLE (Python GUI). To exit, choose "File" menu ⇒ Exit. IDLE is written in Python and is kept under "Lib\idlelib". You can also use "idle.bat", "", "idle.pyw" to start the IDLE.
  • For Mac OS X: [TODO]
Using IDLE

In Python IDLE, you can enter Python statements interactively, similar to interactive Python command-line shell. E.g.,

$ idle3
Python 3.5.2 
[GCC 5.4.0 20160609] on Linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print('Hello, world')
Hello, world
>>> print(2 ** 88)
>>> print(8.01234567890123456789)
>>> print((1+2j) * (3-4j))
Writing Python Script in IDLE

To write a Python script under IDLE, choose "File" menu ⇒ "New File". Enter the above script and save as "" in a directory of your choice (the Python script must be saved with the ".py" extension). To run the script, choose "Run" Menu ⇒ "Run Module". You shall see the outputs on the IDLE console.


  • You can use Alt-P/Alt-N to retrieve the previous/next command in the command history.
  • You can read the Python Manual via the "Help" menu.
Debugging Python Script in IDLE
  1. In the main IDLE console, choose "Debug" ⇒ "Debugger" to pop out the "Debug Control" and turn on the debugging mode.
  2. In the script's edit window, choose "Run" ⇒ "Run Module" to start debugging the script. Tick "Source".
  3. You can set a breakpoint by right-click the source line, choose "Set Breakpoint" (or "Clear Breakpoint").
  4. From the "Debug Control", You can step over (Over), step into (Step), step out (Out), etc.

Eclipse PyDev (Recommended for Web Projects)

There are several Eclipse plug-ins for Python - notably PyDev @

Installing PyDev

To install PyDev plug-in for Eclipse: Launch Eclipse ⇒ Help ⇒ Install New Software ⇒ In "Work With", enter "" ⇒ Add ⇒ Select "PyDev".

Caution: When you are prompted for PyDev certificate, you MUST manually select (check) the certificate, before pressing OK. Otherwise, the installation aborts without any warning.

Configuring PyDev

After the installation, you need to configure Python Interpreter: Launch Eclipse ⇒ Window ⇒ Preference ⇒ PyDev ⇒ Expand "Interpreters" node ⇒ Python Interpreter ⇒ New to configure your Python Interpreter. For examples, in Ubuntu, /usr/bin/python for Python 2.7 or /usr/bin/python3 for Python 3.5; in Windows, python.exe. You can configure many interpreters and choose the desire one for each of your project.

Writing Python Script using PyDev

To start a new Python project: Launch Eclipse ⇒ File ⇒ New ⇒ Project ⇒ PyDev ⇒ PyDev Project ⇒ Enter "Project name", e.g., "HelloPython" ⇒ Choose the "Interpreter" (configured earlier) and "Grammar Version" ⇒ "Finish".

To write a Python script: Right-click on the project ⇒ New ⇒ PyDev Module ⇒ Leave the "Package" empty ⇒ In "Name", enter "hello" ⇒ Key in the following script:

"""First Python module to say Hello"""
def sayHello(name):
    return "Hello, " + name

Running Python Script

To run the script, right-click on the script ⇒ Run As ⇒ Python Run.

Debugging Using PyDev

To debug a script, set a breakpoint by double-clicking on the left-margin of the the desired line ⇒ right-click on the script ⇒ Debug as ⇒ Python run.

You can then trace the execution via "Step Over (current statement) (F5)", "Step Into (function) (F6)", "Step Out (function) (F7)", "Resume (to next breakpoint) (F8)", "Terminate", and etc.

PyCharm (Recommended for Web Projects)






Sublime Text




Python Tools

python-dev - Python development tools

Many of the Python packages require python-dev (Python development tools) to install and build.

(Ubuntu) Installing python-dev
# Python 3
$ sudo apt-get install python3-dev
The following NEW packages will be installed:
  libpython3-dev libpython3.5-dev python3-dev python3.5-dev

$ dpkg --status python3-dev
Version: 3.5.1-3

# Python 2
$ sudo apt-get install python-dev
The following NEW packages will be installed:
  libexpat1-dev libpython-dev libpython2.7-dev python-dev python2.7-dev

$ dpkg --status python-dev
Version: 2.7.11-1

pip - Python Package Manager

  1. pip @
  2. pip documentation @

pip (Python Package Manager) is used for downloading, installing/un-installing, and managing Python packages. It is a replacement for an earlier tool called easy_install.

(Ubuntu) Installing pip

There are two versions of pip: pip2 for Python 2 and pip3 for Python 3. The pip executable could be symlinked to either pip2 or pip3.

# (For Python 3) Install 'pip3'
$ sudo apt-get install python3-pip
$ which pip3
$ ll /usr/bin/pip*
-rwxr-xr-x 1 root root 307 Nov 18 20:28 /usr/bin/pip3*
$ pip3 --version
pip 18.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

# (For Python 2) Install 'pip2'
$ sudo apt-get install python-pip
$ which pip2
$ ll /usr/bin/pip*
-rwxr-xr-x 1 root root 306 xxx xx xx:xx /usr/bin/pip*
-rwxr-xr-x 1 root root 306 xxx xx xx:xx /usr/bin/pip2*
-rwxr-xr-x 1 root root 307 xxx xx xx:xx /usr/bin/pip3*
$ pip2 --version
pip 18.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
$ pip --version
pip 18.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

Take note that pip2 uses Python 2; while pip3 uses Python 3.

Upgrading pip

The apt-get install may not install the latest pip version. You could upgrade pip using pip itself, as follows:

# Python 3
$ sudo pip3 install --upgrade pip
$ pip3 --version

# Python 2
$ sudo pip install --upgrade pip
$ pip --version

Note: If you use 'sudo' in the above commands, the upgrades will be installed in /usr/lib, which will be available system-wide and for all users. On the other hand, if you did not use 'sudo', the upgrades were installed in /home/username/.local/lib, which is meant for the current user.

Using pip

Installing a package:

# Syntax: pip install <package-name>
$ pip install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-13.1.2-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

Showing what files were installed and where:

# Syntax: pip show --files <package-name>
$ pip show --files virtualenv
Name: virtualenv
Version: 15.1.0
Location: /usr/local/lib/python2.7/dist-packages

Installing a package of a specific or minimum version:

# Syntax for installing specific version: pip install <package-name>==<version>
# Syntax for installing minimum version: pip install <package-name>>=<version>
$ pip install virtualenv==13.1.1  # specific version
$ pip install virtualenv>=13.1.1  # minimum version

Installing a package from a URL:

Syntax: pip install <url>
$ pip install

Listing all/out-dated packages:

# List all installed packages
$ pip list

# List out-dated packages
$ pip list --outdated
requests (Current: 2.2.1 Latest: 2.8.1)

Upgrading a package:

# Syntax: pip install --upgrade <package-name>
$ pip install --upgrade requests

Un-Installing a package:

# Syntax: pip uninstall <package-name>
$ pip uninstall requests

Search for a package:

# Syntax: pip search <package-name>
$ pip search requests

Installing package for a user:

# Syntax: pip install --user <package-name>
$ pip install --user requests
   # The package will be installed in $HOME/.local/lib/python2.7/site-packages
Managing a list of packages at specific version numbers

The most important feature of pip is to manage a full list of packages at specific version numbers, so as to synchronize two environments (such as the development and production environments). Furthermore, via the so-called virtual environment (to be described next), each Python application can maintain its own list (and versions) of packages.

To create a list of pip-installed packages in a requirement file:

# Syntax: pip freeze > [list-name.txt]
$ pip freeze > requirements.txt

The requirement file contains a list of packages and their version numbers, e.g.,


You can then duplicate this list of packages in another machine:

# Syntax: pip install -r [list-name.txt]
$ pip install -r requirements.txt

virtualenv - Virtual Environment for Python Application Development

  1. virtualenv @
  2. virtualenv documentation @

A Python "Virtual Environment" is a dedicated directory which contains everything that a Python application needed to run the application in an isolated environment. It includes its own Python interpreter, its own pip and all the pip-installed packages. Via virtual environments, you can test and run different Python applications with different Python versions and packages. You can also easily synchronize the development Python environment and the production Python environment.

(Ubuntu) Installing virtualenv

To install virtualenv via pip2 (for Python 2) or pip3 (for Python 3):

$ sudo pip2 install virtualenv
$ sudo pip3 install virtualenv

# Verifying the installation
$ pip2 show --files virtualenv
$ pip3 show --files virtualenv
Name: virtualenv
Version: 15.1.0
Location: /usr/local/lib/python2.7/dist-packages
$ which virtualenv
$ virtualenv --version
Using virtualenv

Suppose that we start a new Python project in the development environment, and would like the project to run under specific versions of Python Interpreter and packages, which is isolated from the system environment and can be duplicated to the production environment. We can create a virtual environment for this project, which is basically a directory keeping its own Python Interpreter and packages isolated from the system packages and can be duplicated in another environment.

For example, to create a virtual environment in Python 2:

# Goto the project directory
$ cd /path/to/project-directory
# Create a virtual environment called 'myvenv' for this project, 
# which is a directory containing its Python interpreter and packages.
# Syntax: virtualenv <env-folder-name>
$ virtualenv myvenv
New python executable in myvenv/bin/python
Installing setuptools, pip, wheel...done.
$ ll myvenv
drwxrwxr-x 2 xxxx xxxx 4096 xxx xx xx:xx bin/
drwxrwxr-x 2 xxxx xxxx 4096 xxx xx xx:xx include/
drwxrwxr-x 3 xxxx xxxx 4096 xxx xx xx:xx lib/
drwxrwxr-x 2 xxxx xxxx 4096 xxx xx xx:xx local/
-rw-rw-r-- 1 xxxx xxxx   60 Feb  7 23:50 pip-selfcheck.json
$ ll myvenv/bin
-rw-rw-r-- 1 xxxx xxxx    2253 xxx xx xx:xx activate
-rwxrwxr-x 1 xxxx xxxx     242 xxx xx xx:xx pip*
-rwxrwxr-x 1 xxxx xxxx     242 xxx xx xx:xx pip2*
-rwxrwxr-x 1 xxxx xxxx     242 xxx xx xx:xx pip2.7*
-rwxrwxr-x 1 xxxx xxxx 3345416 xxx xx xx:xx python*
lrwxrwxrwx 1 xxxx xxxx       6 xxx xx xx:xx python2 -> python*
lrwxrwxrwx 1 xxxx xxxx       6 xxx xx xx:xx python2.7 -> python*
-rwxrwxr-x 1 xxxx xxxx     249 xxx xx xx:xx wheel*

To create a virtual environment for Python 3, use the -p python3 (or --python=python3) flag, as follows:

$ virtualenv -p python3 myvenv3
New python executable in .../myvenv3/bin/python3
Also creating executable in .../myvenv3/bin/python
Installing setuptools, pip, wheel...done.
$ ll myvenv3/bin
total 4408
-rw-rw-r-- 1 xxxx xxxx    2098 xxx xx xx:xx activate
-rwxrwxr-x 1 xxxx xxxx     241 xxx xx xx:xx pip*
-rwxrwxr-x 1 xxxx xxxx     241 xxx xx xx:xx pip3*
-rwxrwxr-x 1 xxxx xxxx     241 xxx xx xx:xx pip3.5*
lrwxrwxrwx 1 xxxx xxxx       7 xxx xx xx:xx python -> python3*
-rwxrwxr-x 1 xxxx xxxx 4460336 xxx xx xx:xx python3*
lrwxrwxrwx 1 xxxx xxxx       7 xxx xx xx:xx python3.5 -> python3*
-rwxrwxr-x 1 xxxx xxxx    2357 xxx xx xx:xx python-config*
-rwxrwxr-x 1 xxxx xxxx     248 xxx xx xx:xx wheel*

To activate/deactivate a virtual environment:

# Syntax: source [env-name]/bin/activate
$ cd /path/to/project-directory
$ source myvenv/bin/activate
   # The command-prompt changes to "(venv)...$"
   # All Python tasks are carried out inside the virtual environment
   # with the specific version of Python interpreter and packages

# To deactivate virtual environment
(myvenv)....$ deactivate
   # The command-prompt returns to "$"

Take note that all commands issued under a virtual environment run under the Python of the virtual environment. Similarly, pip issued under a virtual environment installs the package inside the virtual environment.


Reference: virtualenvwrapper documentation @

"virtualenvwrapper is a set of extensions to Ian Bicking’s virtualenv tool. The extensions include wrappers for creating and deleting virtual environments and otherwise managing your development work flow, making it easier to work on more than one project at a time without introducing conflicts in their dependencies."

[TODO] more


  1. The Python's mother site @; "The Python Documentation" @; "The Python Tutorial" @; "The Python Language Reference" @