Can You Use Python to Debug an Issue
This section explores tools to understand improve your lawmaking base: debugging, to observe and prepare bugs.
It is non specific to the scientific Python customs, but the strategies that we will employ are tailored to its needs.
2.3.one. Avoiding bugs¶
ii.3.ane.1. Coding all-time practices to avert getting in trouble¶
- Nosotros all write buggy code. Accept information technology. Deal with it.
- Write your code with testing and debugging in heed.
- Go on It Simple, Stupid (KISS).
- What is the simplest affair that could perhaps work?
- Don't Repeat Yourself (Dry).
- Every slice of knowledge must have a single, unambiguous, authoritative representation inside a system.
- Constants, algorithms, etc…
- Effort to limit interdependencies of your code. (Loose Coupling)
- Give your variables, functions and modules meaningful names (not mathematics names)
2.iii.i.2. pyflakes: fast static analysis¶
They are several static analysis tools in Python; to name a few:
- pylint
- pychecker
- pyflakes
- flake8
Here nosotros focus on pyflakes, which is the simplest tool.
- Fast, simple
- Detects syntax errors, missing imports, typos on names.
Another adept recommendation is the flake8 tool which is a combination of pyflakes and pep8. Thus, in addition to the types of errors that pyflakes catches, flake8 detects violations of the recommendation in PEP8 mode guide.
Integrating pyflakes (or flake8) in your editor or IDE is highly recommended, information technology does yield productivity gains.
Running pyflakes on the current edited file¶
You can demark a key to run pyflakes in the current buffer.
-
In kate Menu: 'settings -> configure kate
-
In plugins enable 'external tools'
-
In external Tools', add together pyflakes:
kdialog -- title "pyflakes %f ilename" -- msgbox "$(pyflakes %f ilename)"
-
-
In TextMate
Menu: TextMate -> Preferences -> Avant-garde -> Shell variables, add a beat variable:
TM_PYCHECKER = / Library / Frameworks / Python . framework / Versions / Current / bin / pyflakes
And so Ctrl-Shift-V is binded to a pyflakes report
-
In vim In your .vimrc (binds F5 to pyflakes):
autocmd FileType python let &mp = 'echo "*** running % ***" ; pyflakes %' autocmd FileType tex,mp,rst,python imap <Esc>[xv~ <C-O>:make!^M autocmd FileType tex,mp,rst,python map <Esc>[15~ :make!^One thousand autocmd FileType tex,mp,rst,python set autowrite
-
In emacs In your .emacs (binds F5 to pyflakes):
( defun pyflakes - thisfile () ( interactive ) ( compile ( format "pyflakes %due south " ( buffer - file - name ))) ) ( define - minor - style pyflakes - style "Toggle pyflakes manner. With no argument , this command toggles the mode . Not - null prefix statement turns on the way . Nothing prefix argument turns off the mode . " ;; The initial value . nil ;; The indicator for the manner line . " Pyflakes" ;; The minor mode bindings . '( ([f5] . pyflakes-thisfile) ) ) ( add - hook 'python-mode-hook (lambda () (pyflakes-manner t)))
A type-as-become spell-checker like integration¶
-
In vim
-
Use the pyflakes.vim plugin:
- download the zip file from http://www.vim.org/scripts/script.php?script_id=2441
- extract the files in
~/.vim/ftplugin/python - make sure your vimrc has
filetype plugin indent on
-
Alternatively: employ the syntastic plugin. This can be configured to use
flake8too and likewise handles on-the-fly checking for many other languages.
-
-
In emacs
Use the flymake mode with pyflakes, documented on https://www.emacswiki.org/emacs/FlyMake and included in Emacs 26 and more recent. To actuate it, utilise
M-x(meta-primal then x) and enter flymake-mode at the prompt. To enable it automatically when opening a Python file, add the following line to your .emacs file:( add together - hook 'python-mode-hook ' ( lambda () ( flymake - way )))
2.3.2. Debugging workflow¶
If you do have a non piffling bug, this is when debugging strategies kick in. In that location is no silverish bullet. Yet, strategies help:
For debugging a given problem, the favorable situation is when the problem is isolated in a modest number of lines of code, outside framework or application code, with short modify-run-fail cycles
-
Make it fail reliably. Find a test case that makes the lawmaking neglect every time.
-
Divide and Conquer. One time yous have a declining test case, isolate the failing lawmaking.
- Which module.
- Which function.
- Which line of code.
=> isolate a minor reproducible failure: a test case
-
Change 1 thing at a time and re-run the failing examination case.
-
Use the debugger to sympathise what is going wrong.
-
Take notes and be patient. It may take a while.
Note
Once you lot have gone through this process: isolated a tight piece of code reproducing the bug and fix the bug using this piece of code, add the corresponding code to your test suite.
2.iii.3. Using the Python debugger¶
The python debugger, pdb : https://docs.python.org/library/pdb.html, allows you to inspect your code interactively.
Specifically information technology allows y'all to:
- View the source code.
- Walk up and down the call stack.
- Audit values of variables.
- Modify values of variables.
- Fix breakpoints.
impress
Yeah, print statements do work equally a debugging tool. However to audit runtime, information technology is frequently more efficient to utilise the debugger.
ii.3.three.1. Invoking the debugger¶
Ways to launch the debugger:
- Postmortem, launch debugger after module errors.
- Launch the module with the debugger.
- Call the debugger inside the module
Postmortem¶
State of affairs: You're working in IPython and you lot get a traceback.
Here we debug the file index_error.py . When running it, an IndexError is raised. Type %debug and drop into the debugger.
In [1]: % run index_error . py --------------------------------------------------------------------------- IndexError Traceback (virtually recent phone call last) /home/varoquau/dev/scipy-lecture-notes/advanced/debugging/index_error.py in <module>() vi vii if __name__ == '__main__': ----> viii index_error() ix /dwelling house/varoquau/dev/scipy-lecture-notes/avant-garde/debugging/index_error.py in index_error() iii def index_error(): iv lst = list('foobar') ----> 5 print lst[len(lst)] half-dozen 7 if __name__ == '__main__': IndexError: list alphabetize out of range In [2]: % debug > /domicile/varoquau/dev/scipy-lecture-notes/advanced/debugging/index_error.py(five)index_error() four lst = list('foobar') ----> 5 print lst[len(lst)] vi ipdb> list 1 """Pocket-size snippet to raise an IndexError.""" 2 3 def index_error(): 4 lst = list('foobar') ----> 5 print lst[len(lst)] 6 7 if __name__ == '__main__': 8 index_error() nine ipdb> len(lst) 6 ipdb> print(lst[len(lst)-1]) r ipdb> quit In [3]: Mail-mortem debugging without IPython
In some situations you cannot utilise IPython, for instance to debug a script that wants to exist chosen from the command line. In this instance, you lot tin can call the script with python -g pdb script.py :
$ python -m pdb index_error.py > /home/varoquau/dev/scipy-lecture-notes/advanced/optimizing/index_error.py(1)<module>() -> """Small snippet to raise an IndexError.""" (Pdb) continue Traceback (near recent telephone call final): File "/usr/lib/python2.6/pdb.py", line 1296, in primary pdb._runscript(mainpyfile) File "/usr/lib/python2.6/pdb.py", line 1215, in _runscript self.run(argument) File "/usr/lib/python2.six/bdb.py", line 372, in run exec cmd in globals, locals File "<cord>", line ane, in <module> File "index_error.py", line 8, in <module> index_error() File "index_error.py", line 5, in index_error print lst[len(lst)] IndexError: list index out of range Uncaught exception. Entering post mortem debugging Running 'cont' or 'pace' will restart the plan > /home/varoquau/dev/scipy-lecture-notes/avant-garde/optimizing/index_error.py(5)index_error() -> impress(lst[len(lst)]) (Pdb) Pace-by-step execution¶
Situation: You believe a bug exists in a module only are not certain where.
For instance we are trying to debug wiener_filtering.py . Indeed the code runs, only the filtering does not work well.
-
Run the script in IPython with the debugger using
%run -d wiener_filtering.p:In [1]: % run - d wiener_filtering . py *** Blank or comment *** Blank or comment *** Blank or annotate Breakpoint ane at /abode/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py:4 Notation: Enter 'c' at the ipdb> prompt to start your script. > <string>(i)<module>()
-
Ready a break signal at line 34 using
b 34:ipdb> north > /home/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py(4)<module>() 3 i---> iv import numpy as np v import scipy as sp ipdb> b 34 Breakpoint 2 at /domicile/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py:34
-
Continue execution to next breakpoint with
c(ont(inue)):ipdb> c > /dwelling/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py(34)iterated_wiener() 33 """ ii--> 34 noisy_img = noisy_img 35 denoised_img = local_mean(noisy_img, size=size)
-
Step into code with
northward(ext)anddue south(tep):nextjumps to the next statement in the current execution context, whilestepwill go across execution contexts, i.east. enable exploring within role calls:ipdb> s > /dwelling house/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py(35)iterated_wiener() ii 34 noisy_img = noisy_img ---> 35 denoised_img = local_mean(noisy_img, size=size) 36 l_var = local_var(noisy_img, size=size) ipdb> n > /habitation/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py(36)iterated_wiener() 35 denoised_img = local_mean(noisy_img, size=size) ---> 36 l_var = local_var(noisy_img, size=size) 37 for i in range(3):
-
Step a few lines and explore the local variables:
ipdb> due north > /abode/varoquau/dev/scipy-lecture-notes/advanced/optimizing/wiener_filtering.py(37)iterated_wiener() 36 l_var = local_var(noisy_img, size=size) ---> 37 for i in range(3): 38 res = noisy_img - denoised_img ipdb> print(l_var) [[5868 5379 5316 ..., 5071 4799 5149] [5013 363 437 ..., 346 262 4355] [5379 410 344 ..., 392 604 3377] ..., [ 435 362 308 ..., 275 198 1632] [ 548 392 290 ..., 248 263 1653] [ 466 789 736 ..., 1835 1725 1940]] ipdb> print(l_var.min()) 0
Oh dearest, zilch just integers, and 0 variation. Here is our bug, we are doing integer arithmetic.
Raising exception on numerical errors
When we run the wiener_filtering.py file, the following warnings are raised:
In [2]: % run wiener_filtering . py wiener_filtering.py:40: RuntimeWarning: split by goose egg encountered in divide noise_level = (ane - noise/l_var ) We tin can plow these warnings in exception, which enables u.s. to do post-mortem debugging on them, and find our problem more quickly:
In [three]: np . seterr ( all = 'heighten' ) Out[3]: { 'carve up' : 'print' , 'invalid' : 'print' , 'over' : 'print' , 'under' : 'ignore' } In [4]: % run wiener_filtering . py --------------------------------------------------------------------------- FloatingPointError Traceback (most recent telephone call last) /home/esc/anaconda/lib/python2.7/site-packages/IPython/utils/py3compat.pyc in execfile(fname, *where) 176 else: 177 filename = fname --> 178 __builtin__.execfile(filename, *where) /home/esc/physique-cuso-python-2013/scipy-lecture-notes/advanced/debugging/wiener_filtering.py in <module>() 55 pl.matshow(noisy_face[cut], cmap=pl.cm.gray) 56 ---> 57 denoised_face = iterated_wiener(noisy_face) 58 pl.matshow(denoised_face[cut], cmap=pl.cm.greyness) 59 /dwelling/esc/physique-cuso-python-2013/scipy-lecture-notes/avant-garde/debugging/wiener_filtering.py in iterated_wiener(noisy_img, size) 38 res = noisy_img - denoised_img 39 noise = (res**2).sum()/res.size ---> 40 noise_level = (1 - noise/l_var ) 41 noise_level[noise_level<0] = 0 42 denoised_img += noise_level*res FloatingPointError: divide by cypher encountered in divide Other means of starting a debugger¶
-
Raising an exception equally a poor homo break signal
If yous notice it deadening to note the line number to prepare a break signal, you can simply raise an exception at the point that you desire to inspect and use IPython's
%debug. Note that in this example you cannot step or keep the execution. -
Debugging test failures using nosetests
Y'all can run
nosetests --pdbto drop in post-mortem debugging on exceptions, andnosetests --pdb-failureto inspect test failures using the debugger.In addition, yous tin use the IPython interface for the debugger in nose by installing the olfactory organ plugin ipdbplugin. You tin can than pass
--ipdband--ipdb-failureoptions to nosetests. -
Calling the debugger explicitly
Insert the following line where y'all want to drop in the debugger:
import pdb ; pdb . set_trace ()
Warning
When running nosetests , the output is captured, and thus it seems that the debugger does not work. Simply run the nosetests with the -s flag.
Graphical debuggers and alternatives
- pudb is a good semi-graphical debugger with a text user interface in the console.
- The Visual Studio Code integrated evolution environment includes a debugging way.
- The Mu editor is a elementary Python editor that includes a debugging mode.
2.3.iii.2. Debugger commands and interaction¶
l(list) | Lists the code at the current position |
u(p) | Walk up the call stack |
d(own) | Walk downwards the call stack |
n(ext) | Execute the next line (does not become down in new functions) |
s(tep) | Execute the side by side statement (goes down in new functions) |
bt | Print the telephone call stack |
a | Print the local variables |
!control | Execute the given Python command (by opposition to pdb commands |
Warning
Debugger commands are not Python code
Yous cannot name the variables the way you desire. For instance, if in y'all cannot override the variables in the current frame with the same name: use different names than your local variable when typing code in the debugger.
Getting help when in the debugger¶
Type h or help to access the interactive assist:
ipdb> help Documented commands (type help <topic>): ======================================== EOF bt cont enable jump pdef r tbreak westward a c continue exit l pdoc restart u whatis alias cl d h listing pinfo render unalias where args clear debug assist northward pp run unt b commands disable ignore next q s until break condition downwardly j p quit step upwardly Miscellaneous help topics: ========================== exec pdb Undocumented commands: ====================== retval rv two.3.iv. Debugging segmentation faults using gdb¶
If you have a segmentation fault, you cannot debug it with pdb, as it crashes the Python interpreter earlier information technology can drop in the debugger. Similarly, if y'all have a problems in C code embedded in Python, pdb is useless. For this we plow to the gnu debugger, gdb, available on Linux.
Before we first with gdb, let us add a few Python-specific tools to it. For this we add a few macros to our ~/.gdbinit . The optimal choice of macro depends on your Python version and your gdb version. I take added a simplified version in gdbinit , but feel gratis to read DebuggingWithGdb.
To debug with gdb the Python script segfault.py , we can run the script in gdb equally follows
$ gdb python ... (gdb) run segfault.py Starting plan: /usr/bin/python segfault.py [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Partition error. _strided_byte_copy (dst=0x8537478 "\360\343G", outstrides=4, src= 0x86c0690 <Address 0x86c0690 out of bounds>, instrides=32, Northward=three, elsize=4) at numpy/core/src/multiarray/ctors.c:365 365 _FAST_MOVE(Int32); (gdb) Nosotros get a segfault, and gdb captures information technology for post-mortem debugging in the C level stack (not the Python call stack). We can debug the C call stack using gdb's commands:
(gdb) up # 1 0x004af4f5 in _copy_from_same_shape ( dest =<value optimized out>, src=<value optimized out>, myfunc=0x496780 <_strided_byte_copy>, swap=0) at numpy/core/src/multiarray/ctors.c:748 748 myfunc(dit->dataptr, dest->strides[maxaxis], As you can encounter, correct now, nosotros are in the C code of numpy. We would similar to know what is the Python code that triggers this segfault, so nosotros get up the stack until nosotros hitting the Python execution loop:
(gdb) up # 8 0x080ddd23 in call_function ( f = Frame 0x85371ec, for file /abode/varoquau/usr/lib/python2.6/site-packages/numpy/core/arrayprint.py, line 156, in _leading_trailing (a=<numpy.ndarray at remote 0x85371b0>, _nc=<module at remote 0xb7f93a64>), throwflag=0) at ../Python/ceval.c:3750 3750 ../Python/ceval.c: No such file or directory. in ../Python/ceval.c (gdb) upwards # 9 PyEval_EvalFrameEx ( f = Frame 0x85371ec, for file /home/varoquau/usr/lib/python2.6/site-packages/numpy/core/arrayprint.py, line 156, in _leading_trailing (a=<numpy.ndarray at remote 0x85371b0>, _nc=<module at remote 0xb7f93a64>), throwflag=0) at ../Python/ceval.c:2412 2412 in ../Python/ceval.c (gdb) Once nosotros are in the Python execution loop, we can apply our special Python helper office. For case we can find the corresponding Python lawmaking:
(gdb) pyframe /home/varoquau/usr/lib/python2.6/site-packages/numpy/cadre/arrayprint.py (158): _leading_trailing (gdb) This is numpy code, we need to get upwardly until we find code that we take written:
(gdb) upwardly ... (gdb) up # 34 0x080dc97a in PyEval_EvalFrameEx ( f = Frame 0x82f064c, for file segfault.py, line 11, in print_big_array (small_array=<numpy.ndarray at remote 0x853ecf0>, big_array=<numpy.ndarray at remote 0x853ed20>), throwflag=0) at ../Python/ceval.c:1630 1630 ../Python/ceval.c: No such file or directory. in ../Python/ceval.c (gdb) pyframe segfault.py (12): print_big_array The corresponding code is:
def make_big_array ( small_array ): big_array = stride_tricks . as_strided ( small_array , shape = ( 2e6 , 2e6 ), strides = ( 32 , 32 )) return big_array def print_big_array ( small_array ): big_array = make_big_array ( small_array ) Thus the segfault happens when press big_array[-ten:] . The reason is but that big_array has been allocated with its end outside the program retentiveness.
Annotation
For a list of Python-specific commands defined in the gdbinit, read the source of this file.
Wrap up exercise
The post-obit script is well documented and hopefully legible. It seeks to answer a problem of actual involvement for numerical computing, simply information technology does non piece of work… Tin can you debug information technology?
Python source code: to_debug.py
""" A script to compare different root-finding algorithms. This version of the script is buggy and does non execute. It is your job to find an set these bugs. The output of the script sould look like: Benching 1D root-finder optimizers from scipy.optimize: brenth: 604678 full function calls brentq: 594454 total function calls ridder: 778394 total office calls bifurcate: 2148380 total function calls """ from itertools import production import numpy as np from scipy import optimize FUNCTIONS = ( np . tan , # Dilating map np . tanh , # Contracting map lambda ten : x ** 3 + 1e-iv * ten , # Almost null gradient at the root lambda x : x + np . sin ( 2 * x ), # Non monotonous part lambda x : i.one * x + np . sin ( four * 10 ), # Fonction with several local maxima ) OPTIMIZERS = ( optimize . brenth , optimize . brentq , optimize . ridder , optimize . bisect ) def apply_optimizer ( optimizer , func , a , b ): """ Return the number of function calls given an root-finding optimizer, a role and upper and lower premises. """ return optimizer ( func , a , b , full_output = True )[ 1 ] . function_calls , def bench_optimizer ( optimizer , param_grid ): """ Find roots for all the functions, and upper and lower bounds given and return the full number of function calls. """ return sum ( apply_optimizer ( optimizer , func , a , b ) for func , a , b in param_grid ) def compare_optimizers ( optimizers ): """ Compare all the optimizers given on a grid of a few dissimilar functions all albeit a signle root in nix and a upper and lower premises. """ random_a = - 1.three + np . random . random ( size = 100 ) random_b = . 3 + np . random . random ( size = 100 ) param_grid = product ( FUNCTIONS , random_a , random_b ) print ( "Benching 1D root-finder optimizers from scipy.optimize:" ) for optimizer in OPTIMIZERS : impress ( ' % 20s : % 8i total office calls' % ( optimizer . __name__ , bench_optimizer ( optimizer , param_grid ) )) if __name__ == '__main__' : compare_optimizers ( OPTIMIZERS ) borkholderuposecushers.blogspot.com
Source: https://scipy-lectures.org/advanced/debugging/index.html
Post a Comment for "Can You Use Python to Debug an Issue"