Python Debugging
Python Debugging
Candidates should be aware that debugging is a process that developers follow to track down any
defects in code. The process involves code instrumentation, inspection, and execution to
determine which particular application state doesn’t align with how the code should run.
Any mistakes that occur when using the language are referred to as syntax errors. For example,
the code will not run correctly if a programmer forgets to use a quotation mark or comma in the
code or fails to terminate a vector.
A run-time error in Python occurs if the programming language understands the instructions a
programmer gives it but cannot follow them. The program may be correct syntactically, but there
may be an issue with program execution.
Candidates should know that logic errors are also referred to as semantic errors. These errors
may not cause the program to crash but can cause it to produce the wrong output.
A Python debugger is a program that helps developers determine what’s not working in a
computer program by enabling programmers to set breakpoints. The Python debugger also
supports code evaluation.
A high-level debugging system integrates technologies and source-level debugger tools that
developers use to control execution and set breakpoints.
Candidates may explain that reactive debugging involves noticing a defect in the application,
finding the error, and fixing it. This process involves using a debugging protocol after a
developer has noticed the bug, including using debuggers or a print debugging method.
9. What is a virtualenv?
Virtualenv is also known as a virtual environment. Candidates should know that although Python
can’t distinguish between each version in the site-package directory, virtualenv separates each
project’s environment and ensures there are no cross-dependencies.
An intentional stop is known as a breakpoint. Developers use breakpoints to pause execution and
inspect the application’s internal state during a particular moment. Candidates should know that
developers can also use breakpoints to instruct the debugger to stop the code.
Rubber duck debugging is a theory and process that programmers can follow when debugging
their code. The process involves explaining their program to a rubber duck, working through
each line of code. As they explain the program, the debugging solution will reveal itself.
Single-line comments
Multi-line comments
Docstring comments
In this section, you’ll find eight Python debugging interview questions. Use them to determine
your candidates’ knowledge of debugging tools.
What is a bug?
A bug is an error that causes the program to generate an unexpected output that is different from the expected
output or no output. What are some of the error codes you saw so far?
CE - Compilation error / Syntax error
RE - Run time error
WA - Wrong answer / Logical error
TLE - Time limit exceeded
import ipdb
def main():
x = 5
y = 10
result1 = multiply(x, y)
ipdb.set_trace() #breakpoint created by ipdb
result2 = add(result1, y)
print(result2)
main()
https://www.freecodecamp.org/news/python-debugging-handbook/#common-code-error-messages
https://www.freecodecamp.org/news/python-debugging-handbook/
Table of Contents
1. Common Code Error Messages
SyntaxError: invalid syntax
IndentationError: unexpected indent
NameError: name 'variable' is not defined
AttributeError: 'module' object has no attribute 'attribute_name'
FileNotFoundError: [Errno 2] No such file or directory: 'filename'
IndexError: list index out of range
ImportError: No module named 'module_name'
TypeError
ValueError
2. How to Debug Python Code
3. Foundational Debugging Techniques
Print Statements
Logging
Exception Handling
Assertions
4. Advanced Debugging Techniques
Unit Testing
Interactive Debugger (PDB)
Remote Debugging
5. Performance Debugging
Code Linters and Analyzers
Profiling
6. IDE Features for Debugging
7. Some Additional Tips for Efficient Debugging
8. How to Search for Solutions to Bugs and Errors
Effective Search Strategies
Leveraging Web Resources
You should check the file path and make sure the file exists at
the specified location.
To fix it, make sure that the index being used is within the valid
range of the sequence.
8. TypeError:
This is a common exception in Python that occurs when an
operation or function is applied to an object of an inappropriate
type. Here are some common types of TypeError:
1. TypeError: unsupported operand type(s) for +: 'type1' and
'type2': This error occurs when trying to perform an operation on
two objects with incompatible types. For example, attempting to add a
string and an integer or multiply a list by a string.
2. TypeError: function_name() takes X positional arguments but
Y were given: This error occurs when calling a function with an
incorrect number of arguments. It indicates that the function expects a
specific number of arguments, but a different number is provided.
3. TypeError: 'int' object is not callable: This error occurs
when you try to call an object as if it were a function, but it's not
callable. For example, attempting to call an integer.
9. ValueError:
This type of error occurs when a function receives an argument
of the correct type but with an inappropriate value.
Here's an example:
Logging Levels:
DEBUG: Detailed information, useful for developers during
debugging.
INFO: General information about what's happening in the program.
WARNING: Indicates something unexpected happened, but the
program can still continue.
ERROR: Something went wrong, and the program can't proceed as
planned.
CRITICAL: A very serious error, possibly causing the program to
crash.
Here's an example of logging:
import logging
logging.basicConfig(level=logging.Info)
def example_function(x, y):
logging.debug(f"Input values: x={x}, y={y}")
result = x + y
logging.debug(f"Result: {result}")
return result
result = example_function(3, 7)
# this format includes the timestamp, module name, and log level in each log message.
Note : In larger applications, it's common to use loggers instead
of the root logger directly. This approach allows for more
granular control over logging in different parts of the application.
You can read more about logging and loggers here.
To learn more about logging and loggers in Python, check out
this blog: https://www.samyakinfo.tech/blog/logging-in-python
Exception Handling
Wrap suspicious code blocks with try-except statements to
catch and handle exceptions. This prevents your program from
crashing abruptly, allowing you to gracefully handle errors and
log relevant information.
# Eg.
safe_divide(10, 2)
safe_divide(5, 0)
In this example, the safe_divide function attempts to perform a
division operation. If a ZeroDivisionError occurs (division by
zero), it's caught in the first except block. If any other type of
exception occurs, it's caught in the second except block.
The finally block always executes, regardless of whether an
exception occurred.
Assertions
An assertion is a statement that you add to your code to check
if a certain condition holds true. If the condition is false, it
indicates a bug or an unexpected situation in your program.
# Handling AssertionError
try:
assert x > 0, "x should be greater than zero"
except AssertionError as e:
print(f"Assertion failed: {e}")
In this example, the assert y != 0 checks whether the divisor
(y) is not zero. If it is zero, the assertion fails, and the program
raises an AssertionError with the specified error message.
Considerations When Using Assertions:
Assertions are typically used during development and debugging. In a
production environment, you may choose to disable assertions for
performance reasons. To disable it, use the -O (eg. python -O
script.py ) command-line option or
the PYTHONOPTIMIZE environment variable. The -O(optimize) flag turns
off assert statements.
Assertions are not meant for input validation from users or external
systems. They are more for catching logical errors in your code.
Assertions should be simple conditions that are easy to check and
understand. Avoid complex expressions or side effects.
Advanced Debugging Techniques:
Unit Testing
Unit testing is a software testing methodology where individual
components or functions of a program are tested in isolation to
ensure that they function correctly. In Python, units typically
refer to functions, methods, or classes.
# my_module.py
# test_my_module.py
import unittest
from my_module import add_numbers
class TestAddNumbers(unittest.TestCase):
def test_add_numbers(self):
result = add_numbers(2, 3)
self.assertEqual(result, 5)
if __name__ == '__main__':
unittest.main()
To run the tests, execute the following command in the terminal:
def test_add_numbers():
result = add_numbers(2, 3)
assert result == 5
To run the tests, simply execute:
pytest test_my_module.py
def some_function():
# Setting a breakpoint at line 4
pdb.breakpoint()
print("Hello, World!")
some_function()
b. Running the Script with -m pdb Option:
Alternatively, you can run your Python script with the -m
pdb option, which automatically starts the debugger. For
example:
python -m pdb your_script.py
A
snapshot of Interactive debugger mode in terminal
Basic Commands:
Now, you can interact with the debugger and use various
commands to inspect variables, step through the code, and
identify and fix issues.
n (next): Continue execution until the next line in the current function
is reached. If the current line contains a function call, it will not step
into the called function.
c (continue): Continue execution until the next breakpoint is
encountered.
s (step): Execute the current line of code and stop at the first possible
occasion (either in a function that is called or at the next line in the
current function).
q (quit): Exit the debugger and terminate the program.
break (or b): break [file:]line_number or break
[function_name] Sets a breakpoint at the specified line
number or function. When the program execution reaches the
breakpoint, it will pause, allowing you to inspect variables and step
through the code.
By strategically placing breakpoints and using these
commands, you can effectively debug your Python code and
identify the source of issues in a systematic manner.
Command Functionality
list or list [first[, last]]: Display the source code around the current
list (or l)
line. Optionally, you can specify a range of lines to display.
Debugger Extensions
Consider using third-party debugging tools and extensions,
such as pdbpp, pudb and ipdb, which enhance the functionality of
the built-in PDB debugger.
pdbpp provides additional features such as syntax highlighting,
tab-completion, and better navigation capabilities.
ipdb is an IPython-based debugger, integrating the powerful
features of IPython into the debugging experience. It offers an
interactive and user-friendly interface for debugging Python
code. It Supports IPython magic commands, making it easier to
perform complex debugging tasks.
pudb is a full-screen, console-based visual debugger that
provides syntax highlighting and an interactive and visually
appealing debugging experience. It includes a visual interface
with a code browser, making it easy to navigate through your
code.
To use any of them, replace pdb with the corresponding
debugger you want to use. For eg. import pdb;
pdb.set_trace() with import pdbpp; pdbpp.set_trace() in your
code.
Remote Debugging
Remote debugging refers to the process of debugging code that
is running on a system or server separate from the development
environment. This is commonly used when the application is
deployed on a remote server, in the cloud, or on a different
device.
Visual Breakpoints:
Breakpoints are markers that pause the execution of your
Python program at a specific line of code, allowing you to
inspect variables, evaluate expressions, and understand the
flow of your program at that point.
PyCharm: Simply click on the left margin next to the line number
where you want to set the breakpoint.
Visual Studio Code: Click on the left margin, or use the shortcut F9.
IDLE: You can add the line import pdb; pdb.set_trace() at the
desired location.
Step Into (F7): Moves to the next line of code and enters function
calls if applicable.
Step Over (F8): Executes the current line of code and stops at the
next line, skipping function calls.
Step Out (Shift + F8): Completes the execution of the current
function and stops at the calling function.
Variable Inspection:
Inspecting variables is crucial for understanding how data
changes during program execution. IDEs provide a Variables
panel where you can view the current state of variables, making
it easier to identify bugs. Simply hover over a variable or check
the Variables tab to see its current value.
A snapshot illustrating the process of setting Conditional breakpoint in Python code within PyCharm
Watch Expressions:
Watch expressions allow you to monitor specific variables or
expressions continuously as your program runs. This feature is
beneficial when you want to keep an eye on certain values
without manually inspecting them at each breakpoint.
Here, we'll talk about a couple of these tools – PyLint and mypy
– so you can see how to install them and how they work.
[MESSAGES CONTROL]
disable = missing-docstring
In this example, we enable all checks except for the missing
docstring check. You can tailor the configuration to match your
coding style and project requirements.
flake8
flake8 combines three main tools:
flake8 your_file.py
#Replace your_file.py with the actual name of your Python file.
Similar to PyLint, flake8 can be configured to suit your project's
requirements. You can create a configuration file (usually
named .flake8) in your project's root directory. eg.
[flake8]
max-line-length = 88
extend-ignore = E203, W503
In this example, we set the maximum line length to 88
characters and extend the list of ignored errors.
Black
Black is an opinionated code formatter that automates
formatting decisions for consistent and readable code.
black your_file.py
Black complements traditional linters like PyLint and flake8. You
can use these tools in combination to ensure both code quality
and consistent formatting.
Many popular editors like Visual Studio Code, Atom, and
Sublime Text have extensions or plugins that allow you to use
these Code Linters and Analyzers results directly within the
editor as you write code.
Profiling
Profiling involves analyzing the performance of your code to
identify bottlenecks and areas that can be optimized. Python
provides built-in tools and external libraries for profiling, helping
developers gain insights into their code's execution time and
resource usage.
def example_function():
# Your code here
if __name__ == "__main__":
cProfile.run('example_function()')
This will output a detailed report of function calls, their execution
time, and the percentage of total time spent in each function.
2. profile:
The profile module is similar to cProfile but is implemented in
pure Python. It provides a more detailed analysis of function
calls and can be used when a more fine-grained profiling is
needed.
import profile
def example_function():
# Your code here
if __name__ == "__main__":
profile.run('example_function()')
Both cProfile and profile produce similar outputs, but the
former is generally preferred for its lower overhead.
How to Visualize Profiling Results:
While the built-in modules provide textual reports, visualizing
the results can make it easier to understand and analyze. One
popular tool for this is snakeviz.
Installing snakeviz:
pip install snakeviz
Using snakeviz:
import cProfile
import snakeviz
def example_function():
# Your code here
if __name__ == "__main__":
cProfile.run('example_function()', 'profile_results')
snakeviz.view('profile_results')
This will open a browser window displaying an interactive
visualization of the profiling results.
Line Profiling:
Line profiling allows you to see how much time is spent on each
line of code within a function. The line_profiler module is
commonly used for this purpose.
Installing line_profiler:
pip install line_profiler
Using line_profiler:
def example_function():
# Your code here
if __name__ == "__main__":
profiler = LineProfiler()
profiler.add_function(example_function)
profiler.run('example_function()')
Using memory_profiler:
from memory_profiler import profile
@profile
def example_function():
# Your code here
if __name__ == "__main__":
example_function()
When executed, this will display a line-by-line analysis of
memory usage during the execution of the example_function
Understanding memory usage is crucial for optimizing code.
The memory_profiler module helps in profiling memory
consumption.
While these techniques cover a broad range of debugging
scenarios, it's important to note that the most effective
debugging often involves a combination of these methods.
Additionally, understanding the specific context and the type of
problem you're dealing with will guide you in choosing the most
appropriate technique.
Conclusion
In this debugging handbook, we've explored common error
messages, learned effective search strategies, and discovered
the practical utility of print statements.