Contents

Python
Python Identifier
Lines and Indentation
Multi-Line Statements
Quotation in Python
Comments in Python
Multiple Statement Groups as Suites
Assigning Values to Variables
Multiple Assignment
Print
Standard Data Types
 Python Strings
 Python Lists
 Python Tuples
 Python Dictionary
Type Conversion in Python
Types of Operators
Python Decision Making
Python Loops
Python Date and Time
Python Functions
 Function Arguments
 Scope of Variables
Python Modules and Packages
Python Files Input/ Output
Python File Methods
Python Excel Handling
KeyStrokes
Python Exception Handling
Python Object Oriented
Class Inheritance
Python Regular Expressions
Python Decorators
Multithreading in Python
Multiprocessing in Python
Memory Management
Tesseract-OCR Module in Python
Pywinautogui Module
Selenium Module
Request Module
List Comprehension
Dictionary Comprehension
Lambda Function
Filter Function
Reduce Function
Socket Module

Python
Python is a general-purpose interpreted, interactive, object-oriented, and high-
level programming language. Python is a high-level, interpreted, interactive and
object-oriented scripting language.
 Python is Interpreted: Python is processed at runtime by the
interpreter. You do not need to compile your program before executing it.
 Python is Interactive: You can actually sit at a Python prompt and
interact with the interpreter directly to write your programs.
 Python is Object-Oriented: Python supports Object-Oriented style or
technique of programming that encapsulates code within objects.
 Python is a Beginner's Language: Python is a great language for the
beginner-level programmers and supports the development of a wide
range of applications from simple text processing to WWW browsers to
games.
 Databases: Python provides interfaces to all major commercial
databases.
 GUI Programming: Python supports GUI applications that can be
created and ported to many system calls, libraries and windows systems,
such as Windows MFC, Macintosh, and the X Window system of Unix.
 It supports functional and structured programming methods as well as
OOP.
 It can be used as a scripting language or can be compiled to byte-code
for building large applications.
Difference between scripting and programming language:
Scripting language doesn’t have compiler.
 It provides very high-level dynamic data types and supports dynamic
type checking.
 IT supports automatic garbage collection.
 It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java.

Running Python
 There are three different ways to start Python –

Interactive Interpreter
Integrated Development Environment- Graphical User Interface (GUI) environment as well, if you
have a GUI application on your system that supports Python IDLE.

CPython
CPython is the implementation of the language called “Python” in C. Python is
an interpreted programming language. Hence, Python programmers need
interpreters to convert Python code into machine code. Whereas Cython is a
compiled programming language. The Cython programs can be executed
directly by the CPU of the underlying computer without using any interpreter.

Cython
Cython is designed as a C-extension for Python. The developers can use Cython
to speed up Python code execution. But they can still write and run Python
programs without using Cython. But the programmers have to install both
Python and C-compiler as a pre-requisite to run Cython programs.

Basis Cython CPython

CPython is the default and most

Interpreto Cython is not a Python interpreter widely used interpreter or
implementation of Python.

Cython is a superset of the Python

What? It is the original Python version.
language.

Speed of
Cython is faster. It is slower.
Execution

Cython understands C specifications

C and Understands the code written
with Python and which additionally
Python using python specifications.
supports calling C/C++ functions.

Python Coding Environment

 Repl.it
 Pycharm
 Atom
 Sublime
 Jupyter notebook
 Visual Studio
 pythonanywhere.com
 http://www.pythontutor.com/visualize.html
 Virtualenv – where we can write code in different versions of Python like
2.7 and 3.6 both

Python Identifiers
Name used to identify a variable, function, class, module or other object. An
identifier starts with a letter A to Z or a to z or an underscore (_) followed by
zero or more letters, underscores and digits (0 to 9).
Python does not allow punctuation characters such as @, $, and % within
identifiers. Python is a case sensitive programming language.
Thus, Manpower and manpower are two different identifiers in Python.

Here are naming conventions for Python identifiers −

 Class names start with an uppercase letter. All other identifiers start with
a lowercase letter.
 Starting an identifier with a single leading underscore indicates that the
identifier is private.
 Starting an identifier with two leading underscores indicates a strongly
private identifier.
 If the identifier also ends with two trailing underscores, the identifier is a
language-defined special name.

Lines and Indentation

Provides no braces to indicate blocks of code for class and function definitions
or flow control. Blocks of code are denoted by line indentation, which is rigidly
enforced.
The number of spaces in the indentation is variable, but all statements within
the block must be indented the same amount.

Multi-Line Statements
Statements in Python typically end with a new line. Python does, however, allow
the use of the line continuation character (\) to denote that the line should
continue.
total = item_one + \
item_two

Quotation in Python
Python accepts single ('), double (") and triple (''' or """) quotes to denote string
literals, as long as the same type of quote starts and ends the string.

Comments in Python
All characters after the # or ## and up to the end of the physical line are part
of the comment and the Python interpreter ignores them.
To comment multiple lines in python use three single quotes (‘'’) or three
double quotes (‘’’’’).

Multiple Statement Groups as Suites

A group of individual statements, which make a single code block are
called suites in Python. Compound or complex statements, such as if, while,
def, and class require a header line and a suite.
Header lines begin the statement (with the keyword) and terminate with a
colon ( : ) and are followed by one or more lines which make up the suite.
Assigning Values to Variables
Multiple Assignment
Python allows you to assign a single value to several variables simultaneously.
a = b = c = 1

You can also assign multiple objects to multiple variables.

a,b,c = 1,2,"john"

Swap two numbers:

a = 1, b = 2
a,b = b,a

Print()
end parameter in print():
By default python’s print() function ends with a newline.

Python’s print() function comes with a parameter called ‘end’. By default,

the value of this parameter is ‘\n’, i.e. the new line character. You can
end a print statement with any character/string using this parameter.
print("Welcome to" , end = ' ')
print("GeeksforGeeks", end = ' ')
Output :
Welcome to GeeksforGeeks
print("Python" , end = '@')
print("GeeksforGeeks")
Output :
Python@GeeksforGeeks

Standard Data Types

Python has five standard data types −
 Numbers
 String
 List
 Tuple
 Dictionary

Python supports four different numerical types −

 int (signed integers)
  long (long integers, they can also be represented in octal and hexadecimal)
 float (floating point real values)
 complex (complex numbers)
Python Strings
In python, the string data types are immutable, which means a string value cannot be updated.
>>> phrase = 'how you like me now'
>>> print('Address of phrase is: {}'.format(id(phrase)))
Address of phrase is: 139929793080104
>>> phrase = 'do you feel lucky'
>>> print('Address of phrase is: {}'.format(id(phrase)))
Address of phrase is: 139929792606832
Since a string is immutable, it created a new string object. The memory
addresses do not match.

Subsets of strings can be taken using the slice operator ([ ] and [:] ) with
indexes starting at 0 in the beginning of the string and working their way from
-1 at the end.
str = 'Hello World!'
print(str) # Prints complete string
print(str[0]) # Prints first character of the string
print(str[2:5]) # Prints characters starting from 3rd to 5th
print(str[2:]) # Prints string starting from 3rd character
print(str * 2) # Prints string two times
print(str + "TEST") # Prints concatenated string
print(str[::-1] # prints reverse of string

# Function to reverse a string

def reverse(string):
string = "".join(reversed(string))
return string

s = "Geeksforgeeks"
print (reverse(s))

The reversed() returns the reversed iterator of the given string and then its
elements are joined empty string separated using join(). And reversed order
string is formed.

Indexing

Slicing

Escape Characters
Escape or non-printable characters that can be represented with backslash
notation.
In Python strings, the backslash "\" is a special character, also called the
"escape" character. It is used in representing certain whitespace characters:
"\t" is a tab,

"\n" is a newline, and

"\r" is a carriage return.

Python Lists
A list contains items separated by commas and enclosed within square brackets
([]). One difference between them is that all the items belonging to a list can be
of different data type.
The values stored in a list can be accessed using the slice operator ([ ] and
[:]) with indexes starting at 0 in the beginning of the list and working their way
to end -1.
The plus (+) sign is the list concatenation operator, and the asterisk
(*) is the repetition operator.
list1 = ['physics', 'chemistry', 1997, 2000];

Built-in List Functions & Methods

Python includes the following list functions −

Sr.No Function with Description

.

1 len(list)
length of elements of list.

2 count(list)
count the number of elements of list.

3 Index(element)
Fetches the 1st index of element in the list.

Python has a set of built-in methods that you can use on lists/arrays.

Method Description

append() Adds an element at the end of the list

 clear() Removes all the elements from the list

copy() Returns a copy of the list

count() Returns the number of elements with the specified value

extend() Add the elements of a list (or any iterable), to the end of the
current list

index() Returns the index of the first element with the specified value

insert() Adds an element at the specified position

pop() Removes the element at the specified position

remove() Removes the first item with the specified value

reverse() Reverses the order of the list

sort() Sorts the list

Indexing

Slicing

Python Tuples
Tuples are enclosed within parentheses. A tuple is a sequence
of immutable Python objects.
The main differences between lists and tuples are: Lists are enclosed in brackets
( [ ] ) and their elements and size can be changed, while tuples are enclosed in
parentheses ( ( ) ) and cannot be updated. Tuples can be thought of as read-
only lists.

To write a tuple containing a single value you have to include a comma, even
though there is only one value −
tup1 = (50,);

Changing a Tuple
This means that elements of a tuple cannot be changed once they have been
assigned. But, if the element is itself a mutable data type like list, its nested items
can be changed.

# Changing tuple values

 my_tuple = (4, 2, 3, [6, 5])

# TypeError: 'tuple' object does not support item assignment

# my_tuple[1] = 9

# However, item of mutable element can be changed

my_tuple[3][0] = 9 # Output: (4, 2, 3, [9, 5])
print(my_tuple)

Tuple Uses:
 To represent single set of data.
 To provide easy access to and manipulation of a data set.
 To return multiple values from a method without using out parameters or ByRef
parameters.

Tuples as return values

def f(r):
""" Return (circumference, area) of a circle of radius r """
c = 2 * math.pi * r
a = math.pi * r * r
return (c, a)

 To pass multiple values to a method through a single parameter

Advantages of Tuple over List

Since tuples are quite similar to lists, both of them are used in similar
situations. However, there are certain advantages of implementing a tuple over
a list. Below listed are some of the main advantages:
 We generally use tuples for heterogeneous (different) data types and lists for
homogeneous (similar) data types.
 Since tuples are immutable, iterating through a tuple is faster than with list. So
there is a slight performance boost.
 Tuples that contain immutable elements can be used as a key for a dictionary.
With lists, this is not possible.
 If you have data that doesn't change, implementing it as tuple will guarantee
that it remains write-protected.

Python Dictionary
Python's dictionaries are kind of hash table type.
Each key is separated from its value by a colon (:), the items are
separated by commas, and the whole thing is enclosed in curly braces. An
empty dictionary without any items is written with just two curly braces, like
this: {}.

Keys are unique within a dictionary while values may not be. The values of a
dictionary can be of any type, but the keys must be of an immutable data
type such as strings, numbers, or tuples.
Dictionaries are enclosed by curly braces ({ }) and values can be assigned and
accessed using square braces ([]).

tinydict = {'name': 'john','code':6734, 'dept': 'sales'}

print(tinydict) # Prints complete dictionary
print(tinydict.keys()) # Prints all the keys
print(tinydict.values()) # Prints all the values

Built-in Dictionary Functions & Methods

Python includes the following dictionary functions −

Sr.No. Function with Description

1 cmp(dict1, dict2)
Compares elements of both dict.

2 len(dict)
Gives the total length of the dictionary. This would be equal to the
number of items in the dictionary.

3 str(dict)
Produces a printable string representation of a dictionary

4 type(variable)
Returns the type of the passed variable. If passed variable is dictionary,
then it would return a dictionary type.

Python includes following dictionary methods −

Sr.No. Methods with Description

1 dict.clear()
Removes all elements of dictionary dict

2 dict.copy()
Returns a shallow copy of dictionary dict

3 dict.fromkeys()
 Create a new dictionary with keys from seq and values set to value.

4 dict.get(key, default=None)
For key key, returns value or default if key not in dictionary

5 dict.has_key(key)
Returns true if key in dictionary dict, false otherwise

6 dict.items()
Returns a list of dict's (key, value) tuple pairs

7 dict.keys()
Returns list of dictionary dict's keys

8 dict.setdefault(key, default=None)
Similar to get(), but will set dict[key]=default if key is not already in dict

9 dict.update(dict2)
Adds dictionary dict2's key-values pairs to dict

10 dict.values()
Returns list of dictionary dict's values

Two important points to remember about dictionary keys –

 More than one entry per key not allowed. Which means no duplicate key is
allowed.
 When duplicate keys encountered during assignment, the last assignment
wins.
dict = {'Name': 'Zara','Age':7, 'Name': 'Manni'}
print(“dict['Name']: ", dict['Name'])
When the above code is executed, it produces the following result −
dict['Name']: Manni

(b) Keys must be immutable. Which means you can use strings, numbers or
tuples as dictionary keys but something like ['key'] is not allowed.

Nested Dictionary
nested_dict = { 'dictA': {'key_1': 'value_1'},
'dictB': {'key_2': 'value_2'}}
Here, the nested_dict is a nested dictionary with the dictionary dictA and dictB. They are two
dictionary each having own key and value.
people = {1: {'name': 'John', 'age': '27', 'sex': 'Male'},

2: {'name': 'Marie', 'age': '22', 'sex': 'Female'}}

print(people[1]['name'])

print(people[1]['age'])

print(people[1]['sex'])

Add another dictionary to the nested dictionary

people[4] = {'name': 'Peter', 'age': '29', 'sex': 'Male', 'married': 'Yes'}
print(people[4])

Delete elements from a Nested Dictionary

del people[2]['sex']

Iterating Through a Nested Dictionary

people = {1: {'Name': 'John', 'Age': '27', 'Sex': 'Male'},
2: {'Name': 'Marie', 'Age': '22', 'Sex': 'Female'}}

for p_id, p_info in people.items():

print("\nPerson ID:", p_id)

for key in p_info:

print(key + ':', p_info[key])

Key Points to Remember:

1. Nested dictionary is an unordered collection of dictionary
2. Slicing Nested Dictionary is not possible.
3. We can shrink or grow nested dictionary as need.
4. Like Dictionary, it also has key and value.
5. Dictionary are accessed using key.
Properties comparison between all data types
Properties List Tuple Set Dictionary
Insertion Order Y Y N N
Duplicates Y Y N N
Heterogeneous Y Y Y Y
Index and slicing Y Y N N
Mutable Y N Y Y

Type Conversion in Python

Implicit
Example:
a=5
a = 5.9

Explicit
1. int(a,base) : This function converts any data type to integer. ‘Base’
specifies the base in which string is if data type is string.
2. float() : This function is used to convert any data type to a floating point
number
3. ord() : This function is used to convert a character to integer.
4. hex() : This function is to convert integer to hexadecimal string.
5. oct() : This function is to convert integer to octal string.
6. tuple() : This function is used to convert to a tuple.
7. set() : This function returns the type after converting to set.
8. list() : This function is used to convert any data type to a list type.
9. dict() : This function is used to convert a tuple of order (key,value) into
a dictionary.
10. str() : Used to convert integer into a string.
11. complex(real,imag) : : This function converts real numbers to
complex(real,imag) number.

Types of Operator
Python language supports the following types of operators.
 Assignment Operators: used to assign values to variables
=, +=, -=, *=, /=, %=, **=, //=
o Simple Assigment:
x= 10
print(x)
o Multiple Assignment:
Raj=Sid=Pari=’48’
Print (Raj)
Print(Sid)
Print(Pari)
o Compound Assignment:
a=10
a+=5
print(a)

 Comparison (Relational) Operators : used to compare two values.

==, !=, >,<, <=, >=,
 Arithmetic Operators: used with numeric values to perform common
mathematical operations
+, -, *, /, %(modulus), **(exponential), //(floor division)
 Logical Operators : used to combine conditional statements.
and, or, not
 Bitwise Operators: used to compare (binary) numbers.
 &(Binary AND), | (Binary OR), ^ (Binary XOR), ~ (Binary Ones Complement),
<< (Binary Left Shift), >> (Binary Right Shift)
 Membership Operators: used to test if a sequence is presented in an object.
in, not in
 Identity Operators: used to compare the objects.
is, is not
Checks if the values are located in the same part of memory.
Ex-Number or String Variables shares memory.
List ,Tuple or dictionary doesn’t share memory.
list = [1,2,3]
id(list[0])
id(list[1])
id(list[2])

Python Operator Overloading

Python operators work for built-in classes. But the same operator behaves differently
with different types. For example, the + operator will perform arithmetic addition on
two numbers, merge two lists, or concatenate two strings.
This feature in Python that allows the same operator to have different meaning
according to the context is called operator overloading.

Python Decision Making

Statement Description

if statements An if statement consists of a boolean expression

followed by one or more statements.

if...else statements An if statement can be followed by an optional else

statement, which executes when the boolean
expression is FALSE.
The keyword ‘elif‘ is short for ‘else if’,

nested if statements You can use one if or else if statement inside

another if or else if statement(s).

Python Loops
Loop Type Description
 while loop Repeats a statement or group of statements while a
given condition is TRUE. It tests the condition before
executing the loop body.

for loop Executes a sequence of statements multiple times and

abbreviates the code that manages the loop variable.

nested loops You can use one or more loop inside any another while,
for or do-while loop.

for Statements
words = ['cat', 'window', 'defenestrate']
for w in words:
print(w, len(w))

cat 3
window 6
defenestrate 12

The range() Function

If you do need to iterate over a sequence of numbers, the built-in
function range() comes in handy. It generates arithmetic progressions.
It returns a list.

The range() function accompanies two sets of parameters.

 range(stop)
 stop: It is the no. of integers to generate and starts from zero.
eg. range(3) == [0, 1, 2].
 range([start], stop[, step])
 Start: It is the starting no. of the sequence.
 Stop: It specifies the upper limit of the sequence.
 Step: It is the incrementing factor for generating the
sequence.
 Points to note:
 Only integer arguments are allowed.
 Parameters can be positive or negative.
 The range() function in Python starts from the zeroth index.
for i in range(5):
print(i)
0
1
2
3
4
It is possible to let the range start at another number, or to specify a different
increment (even negative; sometimes this is called the ‘step’):
range(5, 10)
[5,6,7,8,9]

range(0, 10, 3)
[0, 3, 6, 9]
range(10,0,-1)
[10,9,8,7,6,5,4,3,2,1]

range(1,10,2) will create a list of numbers but with a step of 2.

range(10,0,-1) will create a list of numbers in backward direction.

xrange() function
 Used in Python 2
 Produces a generator object
xrange(1,6)
>>xrange(1,6) #generator object

#to get the sequential number, we need to use loop

for i in xrange(1,5):
print(i)

1
2
3
4

 Takes less memory than range()

import sys
print(sys.getsizeof(xrange(100))
>>32

print(sys.getsizeof(range(100))
>>864

Enumerate() function
If you want access to the index of each element within the body of a loop, use
the built-in enumerate function:

animals = ['cat', 'dog', 'monkey']

for idx, animal in enumerate(animals):
print('#%d: %s' % (idx + 1, animal))
# Prints "#1: cat", "#2: dog", "#3: monkey", each on its own line
Iterator
have to implement a class with __iter__() and __next__() method, keep track
of internal states, raise StopIteration when there was no values to be returned
etc.

Generator
 There is a lot of overhead in building an iterator in Python; we have to
implement a class with __iter__() and __next__() method, keep track of
internal states, raise StopIteration when there was no values to be returned
etc.
 This is both lengthy and counter intuitive. Generator comes into rescue in
such situations.
 A generator is a function that returns an object (iterator) which we can
iterate over (one value at a time).
 If a function contains at least one yield statement, it becomes a generator
function. Both yield and return will return some value from a function.
 The difference is that, while a return statement terminates a function
entirely, yield statement pauses the function saving all its states and later
continues from there on successive calls.

def my_gen():
n=1
print(‘This is printed first’)
#Generator function contains yield statements
yield n

n+ = 1
print(‘This is printed second’)
yield n

n+ = 1
print(‘This is printed last’)
yield n

To restart the process we need to create another generator object using something like
a = my_gen().

a is generator object.

#Using for loop

for item in my_gen():
print(item)

Why generators are used in Python?

1. Easy to Implement
Generators can be implemented in a clear and concise way as compared to their iterator class
counterpart.

2. Memory Efficient
A normal function to return a sequence will create the entire sequence in memory
before returning the result. This is an overkill if the number of items in the sequence is
very large.
Generator implementation of such sequence is memory friendly and is preferred since it
only produces one item at a time.

3. Represent Infinite Stream

Infinite streams cannot be stored in memory and since generators produce only one
item at a time, it can represent infinite stream of data.

4. Pipelining Generators
Suppose we have a log file from a famous fast food chain. The log file has a column
(4th column) that keeps track of the number of pizza sold every hour and we want to
sum it to find the total pizzas sold in 5 years.

Assume everything is in string and numbers that are not available are marked as 'N/A'.
A generator implementation of this could be as follows.

with open('sells.log') as file:

pizza_col = (line[3] for line in file)
per_hour = (int(x) for x in pizza_col if x != 'N/A')
print("Total pizzas sold = ",sum(per_hour))

Loop Control / Jump Statements

Control Statement Description

Terminates the loop statement and transfers execution to the

break statement
statement immediately following the loop.

Causes the loop to skip the remainder of its body and

continue statement
immediately retest its condition prior to reiterating.

The pass statement in Python is used when a statement is

pass statement
required syntactically but you do not want any command or
code to execute.
while True:
pass # Busy-wait for keyboard interrupt

Python Closure
def print_msg(msg):
# This is the outer enclosing function
def printer():
# This is the nested function
print(msg)
return printer # returns the nested function
# Now let's try calling this function.
# Output: Hello
another = print_msg("Hello")
another()

The print_msg() function was called with the string "Hello" and the returned
function was bound to the name another. On calling another(), the message was
still remembered although we had already finished executing
the print_msg() function.
This technique by which some data ("Hello in this case) gets attached to the code is
called closure in Python.
This value in the enclosing scope is remembered even when the variable goes out
of scope or the function itself is removed from the current namespace

Python @ Property Decorator

# using property class
class Celsius:
def __init__(self, temperature=0):
self.temperature = temperature
def to_fahrenheit(self):
return (self.temperature * 1.8) + 32
# getter
def get_temperature(self):
print("Getting value...")
return self._temperature

# setter
def set_temperature(self, value):
print("Setting value...")
if value < -273.15:
raise ValueError("Temperature below -273.15 is not possible")
self._temperature = value

# creating a property object

temperature = property(get_temperature, set_temperature)

human = Celsius(37)
print(human.temperature)
print(human.to_fahrenheit())
 human.temperature = -300

When an object is created, the __init__() method gets called. This method has the
line self.temperature = temperature. This expression automatically
calls set_temperature().
Similarly, any access like c.temperature automatically calls get_temperature().
This is what property does.

The @property Decorator

In Python, property() is a built-in function that creates and returns
a property object. The syntax of this function is:

property(fget=None, fset=None, fdel=None, doc=None)

where,

 fget is function to get value of the attribute

 fset is function to set value of the attribute
 fdel is function to delete the attribute
 doc is a string (like a comment)

String Formatting Operator

Operator % is unique to strings and makes up for the pack of having functions
from C's printf() family.
#!/usr/bin/python
print("My name is %s and weight is %d kg!" % ('Zara', 21))
When the above code is executed, it produces the following result −
My name is Zara and weight is 21 kg!

Built-in List Functions & Methods:

Python includes the following list functions −

SN Function with Description

1 cmp(list1, list2) - Compares elements of both lists.

2 len(list) - Gives the total length of the list.

3 max(list)- Returns item from the list with max value.

4 min(list) - Returns item from the list with min value.

 5 list(seq) - Converts a tuple into list.

Python includes following list methods-

SN Methods with Description

1 list.append(obj)-Appends object obj to list

2 list.count(obj) - Returns count of how many times obj occurs in list

3 list.extend(seq) - Appends the contents of seq to list

4 list.index(obj)- Returns the lowest index in list that obj appears

5 list.insert(index, obj)- Inserts object obj into list at offset index

6 list.pop(obj=list[-1]) - Removes and returns last object or obj from list

7 list.remove(obj) - Removes object obj from list

8 list.reverse() - Reverses objects of list in place

9 list.sort([func]) - Sorts objects of list, use compare func if given

Python Date & Time

Import time;

The function time.time() returns the current system time in ticks.

import time; # This is required to include time module.
ticks = time.time()
print("Number of ticks since 12:00am, January 1, 1970:", ticks)

Getting current time

To translate a time instant from a seconds since the epoch floating-point value
into a time-tuple, pass the floating-point value to a function (e.g., localtime)
that returns a time-tuple with all nine items valid.

import time;
localtime = time.localtime(time.time())
print("Local current time :", localtime)

This would produce the following result, which could be formatted in any other
presentable form −
Local current time : time.struct_time(tm_year=2013, tm_mon=7,
tm_mday=17, tm_hour=21, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=198, tm_isdst=0)

Getting formatted time

to get time in readable format is asctime() –A
localtime = time.asctime( time.localtime(time.time()) )
print("Local current time :", localtime)
This would produce the following result −
Local current time : Tue Jan 13 10:17:09 2009

Getting calendar for a month

import calendar
cal = calendar.month(2008, 1)
print("Here is the calendar:")
print(cal)

The time Module

provides functions for working with times and for converting between
representations.

The calendar Module

By default, calendar takes Monday as the first day of the week and Sunday as
the last one. To change this, call calendar.setfirstweekday() function.

Python Functions
Defining a Function
 Function blocks begin with the keyword def followed by the function name and
parentheses ( ( ) ).
 Any input parameters or arguments should be placed within these parentheses. You
can also define parameters inside these parentheses.
 The code block within every function starts with a colon (:) and is indented.
 The statement return [expression] exits a function, optionally passing back an
expression to the caller. A return statement with no arguments is the same as return
None.

Syntax
def functionname( parameters ):
"function_docstring"
function_suite
return [expression]
Calling a Function
Once the basic structure of a function is finalized, you can execute it by calling
it from another function or directly from the Python prompt.
Pass by reference vs value
All parameters (arguments) in the Python language are passed by reference. It
means if you change what a parameter refers to within a function, the change
also reflects in the calling function.

Function Arguments
 Required arguments
 Keyword arguments - When you use keyword arguments in a function call,
the caller identifies the arguments by the parameter name.
def printme( str ):
printme( str = "My string")

 Default arguments - that assumes a default value if a value is not provided in

the function call for that argument.
def printinfo( name, age = 35 ):

 Variable-length arguments –
Arbitrary Arguments
You may need to process a function for more arguments than you specified
while defining the function. These arguments are called variable-
length arguments and are not named in the function definition, unlike required
and default arguments.
def functionname([formal_args,] *var_args_tuple ):
"function_docstring"
function_suite
return [expression]

An asterisk (*) is placed before the variable name that holds the values of all
non-keyword variable arguments. This tuple remains empty if no additional
arguments are specified during the function call.

 Packing and Unpacking-

Unpacking
 We can use * to unpack the list so that all elements of it can be passed
as different parameters.
# A sample function that takes 4 arguments
# and prints the,
def fun(a, b, c, d):
print(a, b, c, d)

# Driver Code
my_list = [1, 2, 3, 4]

# Unpacking list into four arguments

#fun(my_list) #pass list to the function, the call doesn’t work.
fun(*my_list)
Packing
When we don’t know how many arguments need to be passed to a python
function, we can use Packing to pack all arguments in a tuple.

This type of argument syntax doesn’t allow passing a named argument to

the function.
# A Python program to demonstrate use of packing
# This function uses packing to sum unknown number of arguments
def mySum(*args):
sum = 0
for i in range(0, len(args)):
sum = sum + args[i]
return sum

# Driver code
print(mySum(1, 2, 3, 4, 5))
print(mySum(10, 20))
** is used for dictionaries:

Here ** unpacked the dictionary used with it and passed the items
in the dictionary as keyword arguments to the function.
It let us pass N (variable) number of arguments which can be named or
keyworded.

# A Python program to demonstrate packing of dictionary items

using **

def fun(**kwargs):

# kwargs is a dict
print(type(kwargs))

# Printing dictionary items

for key in kwargs:
print("%s = %s" % (key, kwargs[key]))
 # Driver code
fun(name="geeks", ID="101", language="Python")

The Anonymous Functions

These functions are called anonymous because they are not declared in the
standard manner by using the def keyword. You can use
the lambda keyword to create small anonymous functions.
 Lambda forms can take any number of arguments but return just one value in
the form of an expression. They cannot contain commands or multiple expressions.
 An anonymous function cannot be a direct call to print because lambda requires an
expression.
 Lambda functions have their own local namespace and cannot access variables
other than those in their parameter list and those in the global namespace.
 Although it appears that lambda's are a one-line version of a function, they are not
equivalent to inline statements in C or C++, whose purpose is by passing function
stack allocation during invocation for performance reasons.

Syntax
lambda [arg1 [,arg2,.....argn]]:expression
d = lambda x : x *2
print(d(4))
O/P : 8

Scope of Variables
 Global variables
 Local variables

#!/usr/bin/python
total = 0; # This is global variable.
# Function definition is here
def sum( arg1, arg2 ):
# Add both the parameters and return them."
total = arg1 + arg2; # Here total is local variable.

Global Keyword
Global keyword is a keyword that allows a user to modify a variable outside of the
current scope. It is used to create global variables from a non-global scope i.e
inside a function. Global keyword is used inside a function only when we want to do
assignments or when we want to change a variable. Global is not needed for
printing and accessing.

Rules of global keyword:

 If a variable is assigned a value anywhere within the function’s body, it’s
assumed to be a local unless explicitly declared as global.
 Variables that are only referenced inside a function are implicitly global.
 We Use global keyword to use a global variable inside a function.
 There is no need to use global keyword outside a function.

# Python program to modify a global

# value inside a function
x = 15
def change():

# using a global keyword

global x

# increment value of a by 5
x =x +5
print("Value of x inside a function:", x)
change()
print("Value of x outside a function:", x)
Output:
Value of x inside a function: 20
Value of x outside a function: 20

Python Modules and Packages

The import Statement
# Import module support
import support
# Now you can call defined function that module as follows
support.print_func("Zara")

A module is loaded only once, regardless of the number of times it is imported.

The from...import Statement

from fib import fibonacci

This statement does not import the entire module fib into the current namespace; it
just introduces the item fibonacci from the module fib into the global symbol table of
the importing module.

The from...import * Statement:

from modname import *

This provides an easy way to import all the items from a module into the current
namespace; however, this statement should be used sparingly.
Packages in Python
A package is a collection of Python modules, i.e., a package is a directory of Python modules
containing an additional __init__.py file. The __init__.py distinguishes a package from a
directory that just happens to contain a bunch of Python scripts. Packages can be nested to any
depth, provided that the corresponding directories contain their own __init__.py file.

This file can be empty, and it indicates that the directory it contains is a
Python package, so it can be imported the same way a module can be
imported.

The __init__.py file can also decide which modules the package exports as
the API, while keeping other modules internal, by overriding
the __all__ variable.

When you import a module or a package, the corresponding object created by Python is always
of type module. This means that the distinction between module and package is just at the file
system level. Note, however, when you import a package, only variables/functions/classes in
the __init__.py file of that package are directly visible, not sub-packages or modules.
Error! Filename not specified.

For example, in the datetime module, there is a submodule called date. When you import
datetime, it won't be imported. You'll need to import it separately.
>>> import datetime
>>> date.today()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'date' is not defined
>>> from datetime import date
>>> date.today()
datetime.date(2017, 9, 1)

Namespaces and Scoping

The statement global VarName tells Python that VarName is a global variable. Python
stops searching the local namespace for the variable.
Python implements namespaces as dictionaries. There is a name-to-object mapping,
with the names as keys and the objects as values. Multiple namespaces can use the
same name and map it to a different object. Here are a few examples of namespaces:
 Local Namespace: This namespace includes local names inside a function. This
namespace is created when a function is called, and it only lasts until the
function returns.
 Global Namespace: This namespace includes names from various imported
modules that you are using in a project. It is created when the module is
included in the project, and it lasts until the script ends.
 Built-in Namespace: This namespace includes built-in functions and built-in
exception names.
What is Scope?
A name also has a scope that defines the parts of the program where you could
use that name without using any prefix. Just like namespaces, there are also
multiple scopes in a program. Here is a list of some scopes that can exist during the
execution of a program.
 A local scope, which is the innermost scope that contains a list of local names
available in the current function.
 A scope of all the enclosing functions. The search for a name starts from the
nearest enclosing scope and moves outwards.
 A module level scope that contains all the global names from the current
module.
 The outermost scope that contains a list of all the built-in names. This scope
is searched last to find the name that you referenced.

The dir( ) Function

The dir() built-in function returns a sorted list of strings containing the names defined
by a module.
The list contains the names of all the modules, variables and functions that are defined
in a module. The dir() function can also list out all the available attributes
for a module/list/dictionary.

The reload() Function

When the module is imported into a script, the code in the top-level portion of a
module is executed only once.
Therefore, if you want to re-execute the top-level code in a module, you can use
the reload() function. The reload() function imports a previously imported module
again. The syntax of the reload() function is this −
reload(module_name)

Python Files I/O

Reading Keyboard Input
Python provides two built-in functions to read a line of text from standard input,
which by default comes from the keyboard. These functions are −

 raw_input
 input

The raw_input Function

The raw_input([prompt]) function reads one line from standard input and returns
it as a string (removing the trailing newline).
#!/usr/bin/python
str = raw_input("Enter your input: ");
print("Received input is : ", str)

This prompts you to enter any string and it would display same string on the
screen. When I typed "Hello Python!", its output is like this −
Enter your input: Hello Python
Received input is : Hello Python

The input Function

The input([prompt]) function is equivalent to raw_input, except that it assumes
the input is a valid Python expression and returns the evaluated result to you.
#!/usr/bin/python
str = input("Enter your input: ");
print "Received input is : ", str

This would produce the following result against the entered input −
Enter your input: [x*5 for x in range(2,10,2)]
Recieved input is : [10, 20, 30, 40]

Opening and Closing Files

The open Function
Using Python's built-in open() function. This function creates a file object, which
would be utilized to call other support methods associated with it.
Syntax:
file object = open(file_name [, access_mode])

The file Object Attributes

Once a file is opened and you have one file object, you can get various information
related to that file.

Here is a list of all attributes related to file object:

Attribute Description

file.closed Returns true if file is closed, false otherwise.

file.mode Returns access mode with which file was opened.

 file.name Returns name of the file.

file.softspace Returns false if space explicitly required with print, true otherwise.

The close() Method

The close() method of a file object flushes any unwritten information and closes
the file object, after which no more writing can be done.

Syntax:
fileObject.close();

with Command
The shortest way to open a text file is by using “with” command as follows:
with open("file-name", "r") as fp:
fileData = fp.read()
#to print the contents of the file print(fileData)

Reading and Writing Files

The write() Method
The write() method writes any string to an open file. It is important to note that
Python strings can have binary data and not just text.
Syntax:
fileObject.write(string);

The read() Method

The read() method reads a string from an open file. It is important to note
that Python strings can have binary data apart from text data.
Syntax:
fileObject.read([count]);

File Positions
The tell() method tells you the current position within the file; in other
words, the next read or write will occur at that many bytes from the
beginning of the file.
The seek(offset[, from]) method changes the current file position.
The offset argument indicates the number of bytes to be moved.
The from argument specifies the reference position from where the bytes
are to be moved.
If from is set to 0, it means use the beginning of the file as the reference
position and 1 means use the current position as the reference position and
if it is set to 2 then the end of the file would be taken as the reference
position.

Directories in Python
OS Module
The mkdir() Method
The mkdir() method of the os module to create directories in the current
directory.
#!/usr/bin/python
import os
# Create a directory "test"
os.mkdir("test")

The chdir() Method

The chdir() method to change the current directory. The chdir() method takes
an argument, which is the name of the directory that you want to make the
current directory.
#!/usr/bin/python
import os
# Changing a directory to "/home/newdir"
os.chdir("/home/newdir")

The getcwd() Method

The getcwd() method displays the current working directory.
Syntax:
os.getcwd()

The rmdir() Method

The rmdir() method deletes the directory, which is passed as an argument in
the method.
Syntax:
os.rmdir('dirname')

It is required to give fully qualified name of the directory, otherwise it would

search for that directory in the current directory.
List Directory Contents
The listdir() function returns the content of a directory.
example
import os
contents = os.listdir(r"C:\ Users\ r
print(contents)

Creating Subdirectories
import os
os.makedirs("Temp/ temp1/ temp

Renaming a directory/folder
The os.rename() method can rename a folder from an old name to a new
one.
Example
import os
os.rename("Temp","Temp11")

Check whether a file exists using Python

The os.path.isfile(path) return True if path is an existing regular file.
example
import os.path
filename = "my_file.txt"
if(os.path.isfile(filename)):
print("File Exists!!")

If the file "my_file.txt" exist in the current path, it will return true else false.

Check whether a directory/Folder exists using Python

From python 3.4 a newer the pathlib module is recommended to be used. It offers a
more object oriented approach than functions on python 2 os package.
example
from pathlib import Path
dirname = Path("temp")
if(dirname.is_dir()):
print("Directory/ Folder Exists!!"

Renaming and Deleting Files

The rename() method takes two arguments, the current filename and the
new filename.
Syntax:
import os
os.rename(current_file_name, new_file_name)

The remove() Method

You can use the remove() method to delete files by supplying the name of
the file to be deleted as the argument.

Syntax:
os.remove(file_name)

Python File Methods

A file object is created using open function and here is a list of functions which can
be called on this object −

SN Methods with Description

1 file.close()
Close the file. A closed file cannot be read or written any more.

2 file.flush()
Flush the internal buffer, like stdio's fflush. This may be a no-op on some
file-like objects.

3 file.fileno()
Returns the integer file descriptor that is used by the underlying
implementation to request I/O operations from the operating system.

4 file.isatty()
Returns True if the file is connected to a tty(-like) device, else False.

5 file.next()
Returns the next line from the file each time it is being called.

6 file.read([size])
Reads at most size bytes from the file (less if the read hits EOF before
obtaining size bytes).
file.read() #reads all the content of file
file.read(5) # reads 5 characters from the file
7 file.readline([size])
Reads one entire line from the file. A trailing newline character is kept in the
string.
file.readline(5) #reads line number 5

8 file.readlines([sizehint])
Reads until EOF using readline() and return a list containing the lines. If the
optional sizehint argument is present, instead of reading up to EOF, whole
lines totalling approximately sizehint bytes (possibly after rounding up to an
internal buffer size) are read.
file.readlines() #reads upto EOF
file.readlines(5) # gives list with 5 lines of files

9 file.seek(offset[, whence])
Sets the file's current position

10 file.tell()
Returns the file's current position

11 file.truncate([size])
Truncates the file's size. If the optional size argument is present, the file is
truncated to (at most) that size.

12 file.write(str)
Writes a string to the file. There is no return value.
file.write(“hello world”) #amend the content to the file

13 file.writelines(sequence)
Writes a sequence of strings to the file. The sequence can be any iterable
object producing strings, typically a list of strings.
List_of_lines =[ “One line of text here”, “and another line here”, “and yet
another here”, “and so on and so forth”]
file.writelines(List_of_lines) #write multiple lines to a file

Important Statements
 When you open a file for reading, if the file does not exist, an error occurs
 When you open a file for writing, if the file does not exist, a new file is
created
 When you open a file for writing, if the file exists, the existing file is
overwritten with the new file.

Python Excel Handling

Modules
Xlrd, openpyxl, xlsxwriter, pyxlsb, csv

Pywinauto.application

KeyStrokes
Send_Keys
From pywinauto.keyboard import send_keys
def func_execute_keystrokes(keys, repetitions, time_wait=False, delay=1):
try:
repetitions = round(float(repetitions))
keys = keys.lower()
key_shorthand = {'ctrl': '^', 'alt': '%', 'shift': '+'}
hot_keys = {'esc': 'vk_escape'}
keys_array = keys.split('+')
for index, key in enumerate(keys_array):
if key in hot_keys.keys():
keys_array[index] = hot_keys[key]
key = keys_array[index]
if len(key) > 1 and key not in key_shorthand.keys(): # if
keystroke is not single character
keys_array[index] = '{' + key.upper() + '}'
if key in key_shorthand.keys():
keys_array[index] = key_shorthand[key]
final_keys = ''.join(keys_array)
for i in range(repetitions):
if time_wait:
time.sleep(float(delay))
send_keys(final_keys)
log.info('Entered keystroke ' + keys)

except Exception as e:
log.error('Error sending key strokes: ' + str(sys.exc_info()[1]))
return False, e

Python Exceptions Handling

Raise-if
An expression is tested, and if the result comes up false, an exception is raised.

The assert Statement

When it encounters an assert statement, Python evaluates the accompanying
expression, which is hopefully true. If the expression is false, Python raises
an AssertionError exception.

The syntax for assert is −

assert Expression[, Arguments]

Handling an exception
try:
You do your operations here;
......................
except ExceptionI:
If there is ExceptionI, then execute this block.
except ExceptionII:
If there is ExceptionII, then execute this block.
......................
else:
If there is no exception then execute this block.

Here are few important points about the above-mentioned syntax −

 A single try statement can have multiple except statements. This is useful
when the try block contains statements that may throw different types of
exceptions.
 You can also provide a generic except clause, which handles any exception.
 After the except clause(s), you can include an else-clause. The code in the
else-block executes if the code in the try: block does not raise an exception.
 The else-block is a good place for code that does not need the try: block's
protection.

The except Clause with No Exceptions

You can also use the except statement with no exceptions defined as
follows −

try:
You do your operations here;
......................
except:
If there is any exception, then execute this block.
......................
else:
If there is no exception, then execute this block.
This kind of a try-except statement catches all the exceptions that occur.

The except Clause with Multiple Exceptions

You can also use the same except statement to handle multiple exceptions
as follows −

try:
You do your operations here;
......................
except(Exception1[, Exception2[,...ExceptionN]]]):
If there is any exception from the given exception list,
then execute this block.
......................
else:
If there is no exception, then execute this block.

The try-finally Clause

The finally block is a place to put any code that must execute, whether the try-
block raised an exception or not.
try:
You do your operations here;
......................
Due to any exception, this may be skipped.
finally:
This would always be executed.
......................
You cannot use else clause as well along with a finally clause.

try:
fh = open("testfile", "w")
try:
fh.write("This is my test file for exception handling!!")
finally:
print "Going to close the file"
fh.close()
except IOError:
print "Error: can\'t find file or read data"

When an exception is thrown in the try block, the execution immediately

passes to the finally block. After all the statements in the finally block are
executed, the exception is raised again and is handled in
the except statements if present in the next higher layer of the try-
except statement.

Argument of an Exception
A value that gives additional information about the problem. The contents
of the argument vary by exception. You capture an exception's argument by
supplying a variable in the except clause as follows −

try:
You do your operations here;
......................
except ExceptionType as Argument:
You can print value of Argument here...
Raising an Exceptions
Syntax:
raise [Exception [, args [, traceback]]]

Here, Exception is the type of exception (for example, NameError)

and argument is a value for the exception argument. The argument is optional;
if not supplied, the exception argument is None.

User-Defined Exceptions
In the try block, the user-defined exception is raised and caught in the
except block. The variable e is used to create an instance of the
class Networkerror.

class Networkerror(RuntimeError):
def __init__(self, arg):
self.args = arg

So once you defined above class, you can raise the exception as follows −

try:
raise Networkerror("Bad hostname")
except Networkerror,e:
print e.args

Python Object Oriented

Overview of OOP Terminology
 Class
 Class variable: A variable that is shared by all instances of a class. Class
variables are defined within a class but outside any of the class's methods.
 Data member: A class variable or instance variable that holds data
associated with a class and its objects.
 Function overloading: The operation performed varies by the types of
objects or arguments involved.
  Instance variable: A variable that is defined inside a method and belongs
only to the current instance of a class.
 Inheritance: The transfer of the characteristics of a class to other classes
that are derived from it.
 Instance: An individual object of a certain class. An object obj that belongs
to a class Circle, for example, is an instance of the class Circle.
 Instantiation: The creation of an instance of a class.
 Method
 Object: A unique instance of a data structure that's defined by its class. An
object comprises both data members (class variables and instance variables)
and methods.
Everything in Python is an object, and almost everything has attributes and
methods. All functions have a built-in attribute __doc__, which returns the doc
string defined in the function source code.
 Operator overloading: The assignment of more than one function to an
operator.

Creating Classes
The class statement creates a new class definition. The name of the class
immediately follows the keyword class followed by a colon as follows −
class Employee:
'Common base class for all employees'
empCount = 0

def __init__(self, name, salary):

self.name = name
self.salary = salary
Employee.empCount += 1

def displayCount(self):
print("Total Employee %d" % Employee.empCount)

def displayEmployee(self):
print("Name : ", self.name, ", Salary: ", self.salary)

 Elements outside the __init__ method are static elements; they belong to the
class.
empCount = 0
The first method __init__() is a special method, which is called class constructor
or initialization method that Python calls when you create a new instance of this
class.
We declare other class methods like normal functions with the exception that
the first argument to each method is self. Python adds the self argument to the
list for us.
Self is a reference variable which is always pointing to current Object. Within
the python class to refer current object we should use self variable.
The first argument to the constructor and instance method should be self.
Python virtual machine(PVM) is responsible to provide value for self argument
and it is not required provide explicitly.
By using self we can declare and access instance variable.

Constructor
The name should be always: __init__().
Whenever an object is created, constructor is called automatically, it is not required
to call it explicitly.
The main objective: to declare and initialize variables. For every object,
constructor will be executed once.

Creating Instance Objects

To create instances of a class, you call the class using class name and pass
in whatever arguments its __init__ method accepts.

emp1 = Employee("Zara", 2000)

__new__ method
Whenever a class is instantiated __new__ and __init__ methods are
called. __new__ method will be called when an object is created
and __init__ method will be called to initialize the object. In the base
class object, the __new__ method is defined as a static method which
requires to pass a parameter cls. cls represents the class that is needed to
be instantiated, and the compiler automatically provides this parameter at
the time of instantiation.

__init__ cannot return anything

TypeError: __init__() should return None, not 'str'
This TypeError is raised by the handler that calls __init__ method and it
wouldn’t even make sense to return anything from __init__ method since it’s
purpose is just to alter the fresh state of the newly created instance.
 Types of Variables
1. Instance Variables/ Object Level Variable
2. Static Variables/ Class Level Variable
3. Local Variables/ Method Variables

Accessing Attributes
You access the object's attributes using the dot operator with object. Class
variable would be accessed using class name as follows −

emp1.displayEmployee()
print "Total Employee %d" % Employee.empCount

Instead of using the normal statements to access attributes, you can use the
following functions −

 The getattr(obj, name[, default]) : to access the attribute of object.

 The hasattr(obj,name) : to check if an attribute exists or not.

 The setattr(obj,name,value) : to set an attribute. If attribute does not

exist, then it would be created.

 The delattr(obj, name) : to delete an attribute.

hasattr(emp1, 'age') # Returns true if 'age' attribute exists
getattr(emp1, 'age') # Returns value of 'age' attribute
setattr(emp1, 'age', 8) # Set attribute 'age' at 8
delattr(empl, 'age') # Delete attribute 'age'

Types of Methods
1. Instance method
2. Class method
3. Static method

Class Inheritance
Different Types of Inheritance
 Single inheritance.
 Multi-level inheritance.
 Multiple inheritance.
 Multipath inheritance.
 Hierarchical Inheritance.
 Hybrid Inheritance.

 Inheritance
class SubClassName (ParentClass1[, ParentClass2, ...]):
'Optional class documentation string'
class_suite
Example:
#!/usr/bin/python

class Parent: # define parent class

parentAttr = 100
def __init__(self):
print "Calling parent constructor"

def parentMethod(self):
print 'Calling parent method'

def setAttr(self, attr):

Parent.parentAttr = attr

def getAttr(self):
print "Parent attribute :", Parent.parentAttr

class Child(Parent): # define child class

def __init__(self):
print "Calling child constructor"

def childMethod(self):
print 'Calling child method'

c = Child() # instance of child

c.childMethod() # child calls its method
c.parentMethod() # calls parent's method
c.setAttr(200) # again call parent's method
c.getAttr() # again call parent's method

When the above code is executed, it produces the following result −

Calling child constructor

Calling child method
Calling parent method
Parent attribute : 200
 Multilevel Inheritance
 Multiple Inheritance

class Base1:
pass

class Base2:
pass

class MultiDerived(Base1, Base2):

pass
print(MultiDerived.mro())

In the multiple inheritance scenario, any specified attribute is

searched first in the current class. If not found, the search continues
into parent classes in depth-first, left-right fashion without searching
same class twice.
This order is also called linearization of MultiDerived class and the set of
rules used to find this order is called Method Resolution Order (MRO).
Method Resolution Order
It is based on the "C3 superclass linearization" algorithm. This is called a
linearization, because the tree structure is broken down into a linear order.
 The issubclass(sub, sup) boolean function returns true if the given subclass
sub is indeed a subclass of the superclass sup.
 The isinstance(obj, Class) boolean function returns true if obj is an instance of
class Class or is an instance of a subclass of Class

Polymorphism-Overriding Methods
Polymorphism means same function name (but different signatures) being uses for
different types.

In Python, Polymorphism lets us define methods in the child class that have
the same name as the methods in the parent class. In inheritance, the child
class inherits the methods from the parent class. However, it is possible to
modify a method in a child class that it has inherited from the parent class.
This is particularly useful in cases where the method inherited from the
parent class doesn’t quite fit the child class. In such cases, we re-implement
the method in the child class. This process of re-implementing a method in
the child class is known as Method Overriding.

Encapsulation-Data Hiding
An object's attributes may or may not be visible outside the class definition. You
need to name attributes with a double underscore prefix, and those
attributes then are not being directly visible to outsiders.
class JustCounter:
__secretCount = 0

def count(self):
self.__secretCount += 1
print(self.__secretCount)

counter = JustCounter()
counter.count()
counter.count()
print(counter.__secretCount)

When the above code is executed, it produces the following result −

1
2
Traceback (most recent call last):
File "test.py", line 12, in <module>
Print(counter.__secretCount)
AttributeError: JustCounter instance has no attribute '__secretCount'

You can access such attributes as object._className__attrName.

Print(counter._JustCounter__secretCount)

Single Underscore:
 In Interpreter
_ returns the value of last executed expression value in Python
Prompt/Interpreter

 After a name
To avoid such conflict between python keyword and variable we use
underscore after name

 Before a name
Leading Underscore before variable/function/method name indicates to
programmer that It is for internal use only, that can be modified
whenever class want.
Here name prefix by underscore is treated as non-public. If specify from
Import * all the name starts with _ will not import. Python does not
specify truly private so this ones can be called directly from other
modules if it is specified in __all__, we also call it weak Private.
 Python Regular Expressions
A regular expression is a special sequence of characters that helps you match or
find other strings or sets of strings, using a specialized syntax held in a pattern.
The module re provides full support for Perl-like regular expressions in Python.
The re module raises the exception re.error if an error occurs while compiling or
using a regular expression.

Important Functions of re module:

match
This function attempts to match RE pattern to string with optional flags.
To check the pattern at the beginning of the target string, if then returns
match object otherwise None.
Syntax:
import re
re.match(pattern, string, flags=0)

Parameter Description

Pattern This is the regular expression to be matched.

String This is the string, which would be searched to match the

pattern at the beginning of string.

Flags You can specify different flags using bitwise OR (|).

These are modifiers, which are listed in the table below.

Findall()
returns all non-overlapping matches of pattern in string, as a list of strings.
The string is scanned left-to-right, and matches are returned in the order in which they are
found.
find all the matches and returns the list with objects.
l = re.findall(‘[a-z]’,’ab3ffds7djfh’)
print(l)

Fullmatch()
Complete string should match according to the given pattern, if then returns match object
otherwise None.
import re
match = re.fullmatch(pattern, string,)
group(num) or groups()
We use group(num) or groups() function of match object to get matched expression.

Sr.No. Match Object Method & Description

1 group(num=0)
This method returns entire match pattern (or specific subgroup num)

2 groups()
This method returns all matching subgroups in a tuple (empty if there
weren't any)

>>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")

>>> m.group(0) # The entire match
'Isaac Newton'
>>> m.group(1) # The first parenthesized subgroup.
'Isaac'
>>> m.group(2) # The second parenthesized subgroup.
'Newton'
>>> m.group(1, 2) # Multiple arguments give us a tuple.
('Isaac', 'Newton')

groupdict([default])
Return a dictionary containing all the named subgroups of the match, keyed by the
subgroup name. The default argument is used for groups that did not participate in the
match; it defaults to None. For example:

>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcolm

Reynolds")
>>> m.groupdict()
{'first_name': 'Malcolm', 'last_name': 'Reynolds'}

search Function
This function searches for first occurrence of RE pattern within string with
optional flags. If not found returns None.

Syntax:
import re
re.search(pattern, string, flags=0)

Parameter Description
 Pattern This is the regular expression to be matched.

String This is the string, which would be searched to match the

pattern anywhere in the string.

Flags You can specify different flags using bitwise OR (|).

These are modifiers, which are listed in the table below.

Matching Versus Searching

Python offers two different primitive operations based on regular
expressions: match checks for a match only at the beginning of the string,
while search checks for a match anywhere in the string.
sub()

Search and Replace

Syntax:
re.sub(pattern, repl, string, max=0)

This method replaces all occurrences of the RE pattern in string with repl,
substituting all occurrences unless max provided. This method returns modified
string.
substitution or replacement
re.sub(regex,replacement,targetstring)
s = re.sub(‘\d’,’#’,’ab3f6fds7djf0h’)
print(s)

subn()
to find the number of substitution
return type is tuple
t = re.subn(regex,replacement,targetstring)
tuple(resultstring,number of replacements)
t = re.subn(‘\d’,’#’,’ab3f6fds7djf0h’)
print(s)

split()
returns a list with all the elements separated by split elements
l = re.split(‘[.]’,’www.google.com’)
print(l)
Application Areas of Regular Expression:
 Validations
 Pattern Matching application
 Translators like compilers, interpreters, assemblers
 To develop digital circuits
 To develop communication Protocols like TCP/IP

MetaCharacters
Metacharacters are characters that are interpreted in a special way by a RegEx engine.
[] . ^ $ * + ? {} () \ |

[]- Square brackets

Square brackets specifies a set of characters you wish to match.

. - Period
A period matches any single character (except newline '\n').

^ - Caret
The caret symbol ^ is used to check if a string starts with a certain character.

$ - Dollar
The dollar symbol $ is used to check if a string ends with a certain character.

* - Star
The star symbol * matches zero or more occurrences of the pattern left to it.

+ - Plus
The plus symbol + matches one or more occurrences of the pattern left to it.

? - Question Mark
The question mark symbol ? matches zero or one occurrence of the pattern left to it.

{} - Braces
Consider this code: {n,m}. This means at least n, and at most m repetitions of the pattern
left to it.

| - Alternation
Vertical bar | is used for alternation (or operator).

() - Group
Parentheses () is used to group sub-patterns. For example, (a|b|c)xz match any string
that matches either a or b or c followed by xz

\ - Backslash
Backlash \ is used to escape various characters including all metacharacters.
For example,
\$a match if a string contains $ followed by a. Here, $ is not interpreted by a RegEx
engine in a special way.

Character classes:
[abc]= either a or b or c
[^abc] = except a, b and c
[a-z] = any lowercase alphabet symbol
[A-Z] = any uppercase alphabet symbol
[a-zA-Z] = any alphabet symbol
[0-9] = any numeric symbol
[a-zA-Z0-9] = any alpha numeric symbol
[^a-zA-Z0-9] = special character

Predefined Character classes:

\s = space character
\S = Except space character
\d = any digit
\D = except digits
\w = any word character (alpha numeric character)
\W = any character except word (special character)
. = Every Character

Quantifiers
Used to specify the number of occurrences to match
a = exactly one ‘a’
a+ = atleast one ‘a’
a* = any number of a’s including zero number also
a? = atmost one ‘a’
either one or zero number of a’s
a{n} = Exactly n number of a’s
a{m,n} = minimum m number of a’s and maximum n number of a’s
^a = checks whether the given target string starts with a or not
a$ = checks whether the given target string ends with a or not

Python Decorators
we have two different kinds of decorators in Python:

 Function decorators
 Class decorators
A decorator in Python is any callable Python object that is used to modify a function
or a class. A reference to a function "func" or a class "C" is passed to a decorator
and the decorator returns a modified function or class. The modified functions or
classes usually contain calls to the original function "func" or class "C".

Monkey Patching

Multithreading in Python
import threading
t=threading.thread(target=function, args=[,Kwargs])
t.start()
 target: the function to be executed by thread
 args: the arguments to be passed to the target function
 To start a thread, we use start method of Thread class.
A thread is a lightweight process, and multithreading allows us to execute multiple
threads at once. As you know, Python is a multithreaded language. It has a multi-
threading package.

The GIL (Global Interpreter Lock) ensures that a single thread executes at a
time. A thread holds the GIL and does a little work before passing it on to the next
thread. This makes for an illusion of parallel execution. But in reality, it is just
threaded taking turns at the CPU. Of course, all the passing around adds overhead
to the execution.

Pros
 Lightweight - low memory footprint
 Shared memory - makes access to state from another context easier
 Allows you to easily make responsive UIs
 cPython C extension modules that properly release the GIL will run in parallel
 Great option for I/O-bound applications

Cons
 cPython - subject to the GIL
 Not interruptible/killable
  If not following a command queue/message pump model (using
the Queue module), then manual use of synchronization primitives become a
necessity (decisions are needed for the granularity of locking)
 Code is usually harder to understand and to get right - the potential for race
conditions increases dramatically

Multithreaded Priority Queue in Python

A Priority Queue is an extension of the queue with the following properties:
 An element with high priority is dequeued before an element with low priority.
 If two elements have the same priority, they are served according to their
order in the queue.
 import threading

 def my_function():
 print("This is a thread.")

 # Create a thread
 my_thread = threading.Thread(target=my_function)

 # Start the thread
 my_thread.start()

 # Wait for the thread to finish
 my_thread.join()
Events − Events are suitable for simple signaling between threads. If you only need to notify another
thread that a specific event has occurred, events provide a lightweight solution.
Queues − Queues are more versatile and can handle more complex communication between threads.
If you need to exchange data or messages between threads, queues provide a thread-safe and efficient
mechanism.

Multiprocessing in Python
Multiprocessing refers to the ability of a system to support more than one processor
at the same time. The operating system allocates these threads to the processors
improving performance of the system.
# importing the multiprocessing module
import multiprocessing

def print_cube(num):
print("Cube: {}".format(num * num * num))

def print_square(num):
print("Square: {}".format(num * num))

if __name__ == "__main__":
 # creating processes
p1 = multiprocessing.Process(target=print_square, args=(10, ))
p2 = multiprocessing.Process(target=print_cube, args=(10, ))

# starting process 1
p1.start()
# starting process 2
p2.start()

# wait until process 1 is finished

p1.join()
# wait until process 2 is finished
p2.join()

# both processes finished

print("Done!")

Advantages
In multiprocessing, any newly created process will do following:
 run independently
 have their own memory space.
 The threading module uses threads, the multiprocessing module uses
processes. The difference is that threads run in the same memory space,
while processes have separate memory. This makes it a bit harder to
share objects between processes with multiprocessing. Since threads use
the same memory, precautions have to be taken or two threads will write
to the same memory at the same time. This is what the global interpreter
lock is for.
 Spawning processes is a bit slower than spawning threads. Once they are
running, there is not much difference.

Memory Management
Every object in Python has a reference count and a pointer to a type.
 ob_refcnt: reference count
 ob_type: pointer to another type

Python has a private heap space to hold all objects and data structures. Being
programmers, we cannot access it; it is the interpreter that manages it. But with
the core API, we can access some tools. The Python memory manager controls the
allocation.

Garbage Collection
Additionally, an inbuilt garbage collector recycles all unused memory so it can make
it available to the heap space.
Python allows you to inspect the current reference count of an object with
the sys module. You can use sys.getrefcount(numbers), but keep in mind that
passing in the object to getrefcount() increases the reference count by 1.

In any case, if the object is still required to hang around in your code, its reference
count is greater than 0. Once it drops to 0, the object has a specific deallocation
function that is called which “frees” the memory so that other objects can use it.

Destroying Objects (Garbage Collection)

According to Python Official Documentation, you can force the Garbage Collector
to release unreferenced memory with gc.collect().
The garbage collector destroys an orphaned instance and reclaims its space. But
a class can implement the special method __del__(), called a destructor, that
is invoked when the instance is about to be destroyed. This method might be
used to clean up any non-memory resources used by an instance.
def __del__(self):
class_name = self.__class__.__name__
print(class_name, "destroyed")
Time Complexity in Python

Synchronous and Asynchronous

Coroutine
What is the difference between a "generator" and a "coroutine" in Python?
A generator is an iterator that generates values on the fly as needed. It is defined using the yield
keyword and iterates over a "for" loop or calling the next() function. Generators are useful for
generating large sequences of values that may be too large to store in memory.
On the other hand, a coroutine is a special kind of function that can be paused and resumed at
specific points. It is defined using the async def syntax and iterates using an async for loop or
by calling the await function. Coroutines help perform asynchronous operations, such as
network or database I/O, without blocking the main thread of execution.

Asyncio
The "asyncio" library is a built-in library in Python that provides an infrastructure for writing
asynchronous, concurrent, and parallel code. It is designed to help developers write highly
efficient and scalable network servers and clients. Asyncio enables you to write code that can
perform I/O operations without blocking the main thread of execution, which can significantly
improve the performance and responsiveness of your applications.

What does -> mean in Python function definitions?

In Python, the "->" symbol is used to indicate the return type of a function. It is part of the
function definition in a Python 3.5 or later. For example, the following code defines a
function add that takes in two arguments, a and b , and returns the sum of the two arguments.
def add(a: int, b: int) -> int:
return a + b
The -> int at the end of the function definition indicates that the function returns an integer.
This is known as type hinting and is used to help with code readability and debugging. Please
note that this feature is not enforced in Python and it's just used as a hint for developer and
IDEs.

What does __all__ mean in Python?

In Python, __all__ is a list of strings that defines the names that should be imported when from
<module> import * is used. For example:

__all__ = ['foo', 'bar']

def foo():
pass

def bar():
pass

def baz():
pass
If someone writes from my_module import *, only foo and bar will be imported, because they are the
only names listed in __all__. baz will not be imported, because it is not in __all__.

Tesseract-OCR Module in Python

OCR = Optical Character Recognition. In other words, OCR systems transform a
two-dimensional image of text, that could contain machine printed or handwritten
text from its image representation into machine-readable text.
import cv2
import pytesseract

img = cv2.imread('image.jpg')

To avoid all the ways your tesseract output accuracy can drop, you need to make
sure the image is appropriately pre-processed.
This includes rescaling, binarization, noise removal, deskewing, etc.
After preprocessing with the following code
image = cv2.imread('aurebesh.jpg')

gray = get_grayscale(image)
thresh = thresholding(gray)
opening = opening(gray)
canny = canny(gray)
 digitize invoices, PDFs or number plates

Limitations of Tesseract
 The OCR is not as accurate as some commercial solutions available to us.
 Doesn't do well with images affected by artifacts including partial occlusion,
distorted perspective, and complex background.
 It is not capable of recognizing handwriting.
 It may find gibberish and report this as OCR output.
 If a document contains languages outside of those given in the -l LANG
arguments, results may be poor.
 It is not always good at analyzing the natural reading order of documents. For
example, it may fail to recognize that a document contains two columns, and
may try to join text across columns.
 Poor quality scans may produce poor quality OCR.
 It does not expose information about what font family text belongs to.

Pywinautogui Module
PyAutoGUI is a cross-platform GUI automation Python module for human
beings. Used to programmatically control the mouse & keyboard.

Keyboard and Mouse Control

The x, y coordinates used by PyAutoGUI has the 0, 0 origin coordinates in the top
left corner of the screen.

>>> import pyautogui

>>> screenWidth, screenHeight = pyautogui.size() # Returns two integers,
the width and height of the screen. (The primary monitor, in multi-monitor
setups.)
>>> currentMouseX, currentMouseY = pyautogui.position() # Returns two
integers, the x and y of the mouse cursor's current position.
>>> pyautogui.moveTo(100, 150) # Move the mouse to the x, y coordinates
100, 150.
>>> pyautogui.click() # Click the mouse at its current location.
>>> pyautogui.click(200, 220) # Click the mouse at the x, y coordinates
200, 220.
>>> pyautogui.move(None, 10) # Move mouse 10 pixels down, that is, move
the mouse relative to its current position.
>>> pyautogui.doubleClick() # Double click the mouse at the
 >>> pyautogui.moveTo(500, 500, duration=2,
tween=pyautogui.tweens.easeInOutQuad) # Use tweening/easing function to move
mouse over 2 seconds.
>>> pyautogui.write('Hello world!', interval=0.25) # Type with quarter-
second pause in between each key.
>>> pyautogui.press('esc') # Simulate pressing the Escape key.
>>> pyautogui.keyDown('shift')
>>> pyautogui.write(['left', 'left', 'left', 'left', 'left', 'left'])
>>> pyautogui.keyUp('shift')
>>> pyautogui.hotkey('ctrl', 'c')

Display Message Boxes

>>> import pyautogui
>>> pyautogui.alert('This is an alert box.')
'OK'
>>> pyautogui.confirm('Shall I proceed?')
'Cancel'
>>> pyautogui.confirm('Enter option.', buttons=['A', 'B', 'C'])
'B'
>>> pyautogui.prompt('What is your name?')
'Al'
>>> pyautogui.password('Enter password (text will be hidden)')
'swordfish

Screenshot Functions
>>> import pyautogui>>> im2 = pyautogui.screenshot('my_screenshot2.png')

locate where an image is on the screen:

>>> import pyautogui
>>> button7location = pyautogui.locateOnScreen('button.png') # returns (left,
top, width, height) of matching region
>>> button7location
(1416, 562, 50, 41)
>>> buttonx, buttony = pyautogui.center(button7location)
>>> buttonx, buttony
(1441, 582)
>>> pyautogui.click(buttonx, buttony) # clicks the center of where the button
was found

The locateCenterOnScreen() function returns the center of this match

region:

>>> import pyautogui

>>> buttonx, buttony = pyautogui.locateCenterOnScreen('button.png') # returns
(x, y) of matching region
>>> buttonx, buttony
(1441, 582)
>>> pyautogui.click(buttonx, buttony) # clicks the center of where the button
was found

Selenium Module
1. Usage
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://www.python.org")

2. Locating Elements
Selenium provides the following methods to locate elements in a page:
 find_element_by_id
 find_element_by_name
 find_element_by_xpath
 find_element_by_link_text
 find_element_by_partial_link_text
 find_element_by_tag_name
 find_element_by_class_name
 find_element_by_css_selector

To find multiple elements (these methods will return a list):

 find_elements_by_name
 find_elements_by_xpath
 find_elements_by_link_text
 find_elements_by_partial_link_text
 find_elements_by_tag_name
 find_elements_by_class_name
 find_elements_by_css_selector

3. Waits
most of the user action requires some kind of wait before performing it.

Reasons could be many, including but not limited to below.

1. Page is not loaded yet

2. Element to interact with is not available in DOM yet
3. AJAX is still loading and element will be created after AJAX
4. Delay in page response etc…

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "myDynamicElement"))
)
finally:
driver.quit()

 Implicit Wait
An implicit wait directs the WebDriver to poll the DOM for a certain
amount of time (as mentioned in the command) when trying to locate an
element that is not visible immediately. The default value of time that
can be set using Implicit wait is zero. Its unit is in seconds. Implicit wait
remains associated with the web element until it gets destroyed.

driver.implicitly_wait(100)
The timeout in this example is 100 seconds which will trigger if the
target element is not available during that period.
 Explicit Wait
Following are the two Selenium Python classes needed to implement
explicit waits.
 WebDriverWait, and
 Expected Conditions class of the Python.

4. Action Chains
ActionChains are a way to automate low level interactions such as mouse
movements, mouse button actions, key press, and context menu interactions.
This is useful for doing more complex actions like hover over and drag and drop.
Generate user actions.
When you call methods for actions on the ActionChains object, the actions
are stored in a queue in the ActionChains object. When you call perform(),
the events are fired in the order they are queued up.
ActionChains can be used in a chain pattern:

menu = driver.find_element_by_css_selector(".nav")
hidden_submenu = driver.find_element_by_css_selector(".nav #submenu1")

ActionChains(driver).move_to_element(menu).click(hidden_submenu).perform(
)

How to scroll down to the bottom of a page ?

You can use the execute_script method to execute javascript on the loaded page.
So, you can call the JavaScript API to scroll to the bottom or any other position of
a page.

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

How to take screenshot of the current window ?

Use the save_screenshot method provided by the webdriver:

from selenium import webdriver

driver = webdriver.Firefox()
driver.get('http://www.python.org/')
driver.save_screenshot('screenshot.png')
driver.quit()

Exceptions in Selenium
 NoSuchElmentException
 StaleElementReferenceException
 WebDriverException


we usually use Requests, the beloved Python HTTP library, for simple sites;
and Selenium, the popular browser automation tool, for sites that make heavy
use of Javascript. Using Requests generally results in faster and more
concise code, while using Selenium makes development faster on Javascript
heavy sites.

Request Module
Ping a website or portal for information this is called making a request.

Import requests
r = requests.get(‘https://github.com/timeline.json’)
r.status_code
>>200

Get the Content

import requests
r = requests.get('https://github.com/timeline.json')
print r.text
# The Requests library also comes with a built-in JSON decoder,
print r.json

Working with Headers

By utilizing a Python dictionary, you can access and view a server’s response
headers.

r.headers
{
'status': '200 OK',
'content-encoding': 'gzip',
'transfer-encoding': 'chunked',
 'connection': 'close',
'server': 'nginx/1.0.4',
'x-runtime': '148ms',
'etag': '"e1ca502697e5c9317743dc078f67693f"',
'content-type': 'application/json; charset=utf-8'
}

Encoding
Requests will automatically decade any content pulled from a server. But most
Unicode character sets are seamlessly decoded anyway.

print r.encoding
>> utf-8

Custom Headers
To add custom HTTP headers to a request, you must pass them through a
dictionary to the headers parameter.

import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'content-type': 'application/json'}

r = requests.post(url, data=json.dumps(payload),
headers=headers)

Redirection and History

Requests will automatically perform a location redirection when you use the GET
and OPTIONS verbs in Python.

r.history

Make an HTTP Post Request

r = requests.post(http://httpbin.org/post)
But you can also rely on other HTTP requests too, like PUT, DELETE, HEAD,
and OPTIONS.

r = requests.put("http://httpbin.org/put")
r = requests.delete("http://httpbin.org/delete")
r = requests.head("http://httpbin.org/get")
 r = requests.options("http://httpbin.org/get")

import requests, json

github_url = "https://api.github.com/user/repos"
data = json.dumps({'name':'test', 'description':'some test repo'})
r = requests.post(github_url, data, auth=('user', '*****'))
print r.json

Errors and Exceptions

 If there is a network problem like a DNS failure, or refused connection the
Requests library will raise a ConnectionError exception.
 With invalid HTTP responses, Requests will also raise an HTTPError exception,
but these are rare.
 If a request times out, a Timeout exception will be raised.
 If and when a request exceeds the preconfigured number of maximum
redirections, then a TooManyRedirects exception will be raised.
Any exceptions that Requests raises will be inherited from the
requests.exceptions.RequestException object.

REST API Error Codes

 1xx: Informational - Communicates transfer protocol-level information
 2xx: Success -Indicates that the client’s request was accepted successfully.
 200 - OK
 3xx: Redirection - Indicates that the client must take some additional action in
order to complete their request.
 4xx: Client Error - This category of error status codes points the finger at
clients.(Client error Response)
 400 - Bad Request
 401 – Unauthorized >> Problem with client’s credential.
 403 – Forbidden – Permission is not given to client.
 404 – Not Found – REST API can’t map client’s URI to a resource.
 406 – User requested for a valid resource, however the request included a
special Accept- header that indicates to the server a valid response can only
contain certain types of information.
 5xx: Server Error - The server takes responsibility for these error status codes.
 500 - Internal Server Error

Other Imp Points

How do you create a dictionary which can preserve the order of pairs?
Python 2.7. introduced a new “OrderDict” class in the “collections” module and it provides the
same interface like the general dictionaries but it traverse through keys and values in an
ordered manner depending on when a key was first inserted.
Eg:
from collections import OrderedDict
d = OrderDict([('Company-id':1),('Company-Name':'Intellipaat')])
d.items() # displays the output as: [('Company-id':1),('Company-Name':'Intellipaat')]

What are the commands that are used to copy an object in Python?

The command that is used to copy an object in python includes:

- copy.copy() function: This makes a copy of the file from source to destination. It
returns a shallow copy of the parameter that is passed.
- copy.deepcopy(): This also creates a copy of the object from source to destination. It
returns a deep copy of the parameter that is passed to the function.
The dictionary consists of all the objects and the copy() method which is used as:

newdict = olddict.copy()
The assignment statement doesn’t copy any object but it creates a binding between the
target and the object that is used for the mutable items. Copy is required to keep a copy
of it using the modules that is provided to give generic and shallow operations.

The String.format() method

template.format(p0, p1, ..., k0=v0, k1=v1, ...)

The template is a string containing a mixture of one or more format

codes embedded in constant text. The format method uses its arguments to
substitute an appropriate value for each format code in the template.

Using Lists as Queues

It is also possible to use a list as a queue, where the first element added is the first
element retrieved (“first-in, first-out”) however, lists are not efficient for this purpose.
While appends and pops from the end of list are fast, doing inserts or pops from the
beginning of a list is slow (because all of the other elements have to be shifted by one).

To implement a queue, use collections.deque which was designed to have fast appends
and pops from both ends.

from collections import deque

queue = deque(['Eric','John'])
queue.append('Terry')
print(queue)
queue.popleft()
print(queue)

List Comprehensions
It can be used to construct lists in a very natural, easy way, like a mathematician is
used to do.
S = {x² : x in {0 ... 9}}
V = (1, 2, 4, 8, ..., 2¹²)
M = {x | x in S and x even}
>>> S = [x**2 for x in range(10)]
>>> V = [2**i for i in range(13)]
>>> M = [x for x in S if x % 2 == 0]
>>>
>>> print(S); print(V); print(M)

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096]
[0, 4, 16, 36, 64]

Note: Lines beginning with ">>>" and "..." indicate input to Python (these are the default prompts of
the interactive interpreter). Everything else is output from Python.

The interesting thing is that we first build a list of non-prime numbers, using a single
list comprehension, then use another list comprehension to get the "inverse" of the
list, which are prime numbers.

>>> noprimes = [j for i in range(2, 8) for j in range(i*2, 50, i)]

>>> primes = [x for x in range(2, 50) if x not in noprimes]
>>> print primes

[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]

Lists can contain any type of elements, including strings, nested lists and functions.
You can even mix different types within a list.
The following works on a list of strings and produces a list of lists. Each of the sublists
contains two strings and an integer.

>>> words = 'The quick brown fox jumps over the lazy dog'.split()
>>> print words

['The', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog']

>>>
 >>> stuff = [[w.upper(), w.lower(), len(w)] for w in words]
>>> for i in stuff:
... print i
...

['THE', 'the', 3]
['QUICK', 'quick', 5]
['BROWN', 'brown', 5]
['FOX', 'fox', 3]
['JUMPS', 'jumps', 5]
['OVER', 'over', 4]
['THE', 'the', 3]
['LAZY', 'lazy', 4]
['DOG', 'dog', 3]

>>>
>>> stuff = map(lambda w: [w.upper(), w.lower(), len(w)], words)
>>> for i in stuff:
... print i
...

['THE', 'the', 3]
['QUICK', 'quick', 5]
['BROWN', 'brown', 5]
['FOX', 'fox', 3]
['JUMPS', 'jumps', 5]
['OVER', 'over', 4]
['THE', 'the', 3]
['LAZY', 'lazy', 4]
['DOG', 'dog', 3]

Dictionary Comprehensions
Zip
#dictionary comprehension
names = ['Bruce','Clark','Peter','Logan']
heros = ['Batman','Superman','Spiderman','Wolverine']
#print(list(zip(names,heros)))
print(dict(zip(names,heros)))
...
{'Peter': 'Spiderman', 'Bruce': 'Batman', 'Logan': 'Wolverine',
'Clark': 'Superman'}

Now it's time to use zip. First, we zip the lists and loop
through them in parallel like this:
>>> list(zip(keys,values))
[('a', 1), ('b', 2), ('c', 3)]
>>> D2 = {}
>>> for (k,v) in zip(keys, values):
... D2[k] = v
...
>>> D2
{'a': 1, 'b': 2, 'c': 3}

When we want to initialize a dict from keys, we do this:

>>> D = dict.fromkeys(['a','b','c'], 0)
>>> D
{'a': 0, 'c': 0, 'b': 0}

Lambda Function
Anonymous function that are constructed at runtime.
A simple 1-line function that do not use def or return key keywords. These are
implicit.
#double
def double(x):
return x*2
lambda x: 2*x
x is parameter and (2*x) is the return.
#add x and y
def add(x,y):
return x+y
lambda x,y: x+y
#max of x,y
def max(x,y):
if x>y:
return x
else:
return y
print(max(8,5))

max = lambda x,y : x if x>y else y

print(max(8,5))

Map Function
Apply the same function to each element of a sequence and returns the modified
list.

Map
List,[m,n,p] New list, [f(m), f(n), f(p)]
Function,f()

The above example also demonstrates that you can do exactly the same thing
with map() and a lambda function. However, there are cases when you cannot
use map()and have to use a list comprehension instead, or vice versa. When you
can use both, then it is often preferable to use a list comprehension, because this is
more efficient and easier to read, most of the time.

You cannot use list comprehensions when the construction rule is too complicated to be
expressed with "for" and "if" statements, or if the construction rule can change
dynamically at runtime. In this case, you better use map() and / or filter() with an
appropriate function. Of course, you can combine that with list comprehensions.
#print[16,9,4,1]
def square(lst1):
lst2 = []
for num in lst1:
lst2.append(num**2)
return lst2
print(square[4,3,2,1])

n=[4,3,2,1]
print(list(map(lambda x : x**2, n)))
or
print(list(map(square,n)))
Function,f() = lambda x : x**2

List = n
#list comprehension
print([x**2 for x in n])

Filter Function
Filter items out of sequence and returns filtered list.
Filter
List,[m,n,p] New list, [m,n]
Condition,c()
If(m == condition)
#print[4,3]
def over_two(lst1):
lst2 = [x for x in lst1 if x>2]
return lst2

print(over_two([4,3,2,1]))
n[4,3,2,1]
print(list(filter(lambda x: x>2,n)))

Condition,c() = lambda x: x>2

List = n
#List Comprehension
print([x for x in n if x>2])

Reduce Function
Applies same operation to items of a sequence and uses the result of operation as
first param of next operation and returns an item, not a list.
Reduce
List,[m,n,p] f(f(m,n),p)
Function,f()
#print 24
def mult(lst):
prod = lst[0]
for i in range(1,len(lst)):
prod = prod * lst[i]
return prod
print(mult([4,3,2,1]))

#reduce function
n = [4,3,2,1]
print(list(reduce(lambda x,y : x*y,n)))

Function Annotations
Function annotations are completely optional metadata information about the types used by
user-defined functions.
Annotations are stored in the __annotations__ attribute of the function as a dictionary and have
no effect on any other part of the function. Parameter annotations are defined by a colon after
the parameter name, followed by an expression evaluating to the value of the annotation.
Return annotations are defined by a literal ->, followed by an expression, between the
parameter list and the colon denoting the end of the def statement.
def f(ham: str, eggs: str = 'eggs') -> str:
print("Annotations:", f.__annotations__)
print("Arguments:", ham, eggs)
return ham + ' and ' + eggs
...
>>> f('spam')
Annotations: {'ham': <class 'str'>, 'return': <class 'str'>,
'eggs': <class 'str'>}
Arguments: spam eggs
'spam and eggs'

Python Built-In Exceptions

Exception Cause of Error

AirthmeticError For errors in numeric calculation.

AssertionError If the assert statement fails.

When an attribute assignment or the

AttributeError reference fails.

If there is no input or the file pointer is at

EOFError EOF.

Exception It is the base class for all exceptions.

For errors that occur outside the Python

EnvironmentError environment.

It occurs when the floating point operation

FloatingPointError fails.

GeneratorExit If a generator’s <close()> method gets called.

It occurs when the imported module is not

ImportError available.

IOError If an input/output operation fails.

IndexError When the index of a sequence is out of range.

If the specified key is not available in the

KeyError dictionary.

When the user hits an interrupt key (Ctrl+c or

KeyboardInterrupt delete).

MemoryError If an operation runs out of memory.

 When a variable is not available in local or
NameError global scope.

NotImplementedErro
r If an abstract method isn’t available.

OSError When a system operation fails.

It occurs if the result of an arithmetic

OverflowError operation exceeds the range.

When a weak reference proxy accesses a

ReferenceError garbage collected reference.

If the generated error doesn’t fall under any

RuntimeError category.

It is a base class for all built-in exceptions

StandardError except <StopIteration> and <SystemExit>.

The <next()> function has no further item to

StopIteration be returned.

SyntaxError For errors in Python syntax.

IndentationError It occurs if the indentation is not proper.

TabError For inconsistent tabs and spaces.

SystemError When interpreter detects an internal error.

SystemExit The <sys.exit()> function raises it.

When a function is using an object of the

TypeError incorrect type.

If the code using an unassigned reference

UnboundLocalError gets executed.

UnicodeError For a Unicode encoding or decoding error.

ValueError When a function receives invalid values.

If the second operand of division or modulo

ZeroDivisionError operation is zero.
 Urilib Module

Socket Module
Socket programming is a way of connecting two nodes on a network to
communicate with each other. One socket(node) listens on a particular port at an
IP, while other socket reaches out to the other to form a connection. Server forms
the listener socket while client reaches out to the server.

In simpler terms there is a server and a client.

import socket
import sys

# Create a TCP/IP socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Connect the socket to the port where the server is listening

server_address = ('localhost', 10000)
sock.connect(server_address)

Here we made a socket instance and passed it two parameters. The first
parameter is AF_INET and the second one is SOCK_STREAM. AF_INET
refers to the address family ipv4. The SOCK_STREAM means connection
oriented TCP protocol.

After the connection is established, data can be sent through

the socket with sendall() and received with recv(), just as in the server.

try:

# Send data
message = 'This is the message. It will be repeated.'
print >>sys.stderr, 'sending "%s"' % message
sock.sendall(message)

# Look for the response

amount_received = 0
amount_expected = len(message)
 while amount_received < amount_expected:
data = sock.recv(16)
amount_received += len(data)
print >>sys.stderr, 'received "%s"' % data

finally:
print >>sys.stderr, 'closing socket'
sock.close()
When the entire message is sent and a copy received, the socket is closed to free up the port.

For sending data the socket library has a sendall function. This function allows you to send data
to a server to which the socket is connected and server can also send data to the client using
this function.

A simple server-client program :

Server :
A server has a bind() method which binds it to a specific ip and port so that it can listen
to incoming requests on that ip and port. A server has a listen() method which puts the
server into listen mode. This allows the server to listen to incoming connections. And
last a server has an accept() and close() method. The accept method initiates a
connection with the client and the close method closes the connection with the client.

# first of all import the socket library

import socket

# next create a socket object

s = socket.socket()
print("Socket successfully created")

# reserve a port on your computer in our case it is 12345 but it can be

#anything
port = 12345

# Next bind to the port we have not typed any ip in the ip field
# instead we have inputted an empty string this makes the server listen to
# requests coming from other computers on the network
s.bind(('', port))
print("socket binded to %s" %(port))

# put the socket into listening mode

s.listen(5)
print("socket is listening")

# a forever loop until we interrupt it or an error occurs

while True:

# Establish connection with client.

c, addr = s.accept()
print('Got connection from', addr)
 # send a thank you message to the client.
c.send('Thank you for connecting')

# Close the connection with the client

c.close()

 First of all we import socket which is necessary.

 Then we made a socket object and reserved a port on our pc.
 After that we binded our server to the specified port. Passing an empty string
means that the server can listen to incoming connections from other computers as
well. If we would have passed 127.0.0.1 then it would have listened to only those
calls made within the local computer.
 After that we put the server into listen mode.5 here means that 5 connections are
kept waiting if the server is busy and if a 6th socket trys to connect then the
connection is refused.
 At last we make a while loop and start to accept all incoming connections and
close those connections after a thank you message to all connected sockets.

Client
# Import socket module
import socket

# Create a socket object

s = socket.socket()

# Define the port on which you want to connect

port = 12345

# connect to the server on local computer

s.connect(('127.0.0.1', port))

# receive data from the server

print s.recv(1024)
# close the connection
s.close()

 First of all we make a socket object.

 Then we connect to localhost on port 12345 (the port on which our server runs)
and lastly we receive data from the server and close the connection.
 Now save this file as client.py and run it from the terminal after starting the server
script.

Paramiko
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
conn = ssh.connect(hostname=server, username=username, password=password)
channel = ssh.invoke_shell()
channel.send('1\n')
channel.recv(9999)

ssh.close()

Python Interview Questions

What are the modules used in Python?
os, sys, random, math, copy, operator, collections, re, email, logging, traceback, xml,
config, cgi, ocr, ctypes, shutil, zipfile, string, glob, requests, datetime, win32com.client.
xrange() over range()?
 Xrange returns the xrange object while range returns the list, and uses the same
memory and no matter what the range size is.
 In python 3, range() does what xrange() used to do and xrange() does not exist. If you
want to write code that will run on both Python 2 and Python 3, you can't
use xrange().
 range() can actually be faster in some cases - eg. if iterating over the same
sequence multiple times. xrange() has to reconstruct the integer object every time,
but range() will have real integer objects. (It will always perform worse in terms of
memory however)
 xrange() isn't usable in all cases where a real list is needed. For instance, it
doesn't support slices, or any list methods.

What is PEP 8?
PEP 8 is a coding convention, a set of recommendation, about how to write your Python
code more readable. Python Style guide
https://pep8.org
What is pickling and unpickling?
Pickle module accepts any Python object and converts it into a string representation
and dumps it into a file by using dump function, this process is called pickling.
Python Object String representation (Byte Stream)
  dumps() – This function is called to serialize an object hierarchy.

While the process of retrieving original Python objects from the stored string
representation is called unpickling.
String represntation (Byte Stream) Python Object
 loads() – This function is called to de-serialize a data stream.

What is the difference between ValueError and TypeError?

>>>int("dog")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: 'dog'

a, b, c, d = [3, 4, 5]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: need more than 3 values to unpack
>>> myVar = 5
>>> list = []
>>> list.remove(myVar)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

What are the tools that help to find bugs or perform static analysis?
PyChecker is a static analysis tool that detects the bugs in Python source code and warns
about the style and complexity of the bug. Pylint is another tool that verifies whether
the module meets the coding standard.

Random Module
Explain how can you generate random numbers in Python?
To generate random numbers in Python, you need to import command as
import random
random.randint(a,b)
This returns a random floating point number in the range (a,b).
 random() - 0 to 1 (not inclusive)
  Randint(x,y) - inclusive
 Uniform(x,y) – not inclusive
 randrange()-
randrange(start,stop,step)
randdrange(10) generates a number from 0 to 9
randrange(1,11)  generates a number from 1 to 10
randrange(1,11,2)  generates a number 1,3,5,7,9 randomly
 shuffle(list) - shuffles in place and doesn't return anything
 sample(population,k)- Return a k length list of unique elements chosen from the
population sequence. Used for random sampling without replacement.
from random import sample
words = ['red', 'adventure', 'cat', 'cat']
newwords = sample(words, len(words)) # Copy and shuffle
print(newwords) # Possible Output: ['cat', 'cat', 'red', 'adventure']

What is the command to shuffle a list and to randomly select an element in a list?
list = [‘apple’,’banana’,’grapes’]
random.shuffle(list)
random.choice(list)

How to get the type of an object, without using type() function?

a= 12
print(type(a))
print(a.__class__)

output:
<class 'int'>
<class 'int'>

What are Python Decorators?

Python decorator is a specific change that we make in python syntax to alter functions
easily.
How to convert a number to string in Python?
To convert, e.g., the number 144 to the string '144', use the built-in function str(). If you
want a hexadecimal or octal representation, use the built-in functions hex() or oct().
What is the difference between list and Tuple?
The difference between list and Tuple is that list is mutable while tuple is not. Tuple can
be hashed for e.g As a key for dictionaries.
How do I copy a file?
The shutil module contains a copyfile() function to copy a file from source to
destination.
import shutil
shutil.copyfile('/path/to/file', '/path/to/other/phile')

What are the benefits of using Python?

The benefits of Python are that it is simple and easy, portable, extensible, build in data
structure and it is an open source.
What are Dict and List comprehension?
They are syntax constructions to ease the creation of a Dictionary or List based on
existing iterable.
What is pass in Python?
Pass means no-operation Python statement, or in other words it is a place holder in
compound statement, where there should be a blank left and nothing has to be written
there.
In Python what are iterators?
In Python, iterators are used to iterate a group of elements, containers like list.
What is unittest in Python?
A unit testing framework in Python is known as unittest.
It supports sharing of setups, automation testing, shutdown code for tests, aggregation
of tests into collections etc.
In Python what is slicing?
A mechanism to select a range of items from sequence types like list, tuple, strings etc is
known as slicing.
What are generators in python?
The way of implementing iterators is known as generators.
It is a normal function except that it yields expression in the function.
What is docstring in Python?
A Python documentation string is known as docstring, it is a way of documenting Python
functions, modules and classes.
What is map?
Map executes the function given as the first argument on all the elements of the
iterable given as the second argument.
Describe python usage in web programming?
Python is used perfectly for web programming and have many special features to make
it easy to use.
Why is Python 2.7 preferred over the latest versions?
Since it supports all the packages that data scientists will need for data analysis,
including scipy, numpy, matplotlib, sklearn, pyspark, etc. Most of the companies who
use Python are using 2.7.x right now.
Difference:
1. Print: print function in Python 2.x is replaced by print() function in
Python 3.x
2. Division with Integers: If we are porting our code or executing the
python 3.x code in python 2.x, it can be dangerous if integer division
changes go unnoticed (since it doesn’t raise any error). It is
preferred to use the floating value (like 7.0/5 or 7/5.0) to get the
expected result when porting our code.
3. In Python 3, there is only one type “int” for all type of integers.
In Python 2.7. there are two separate types “int” (which is 32 bit)
and “long int” that is same as “int” of Python 3.x, i.e., can store
arbitrarily large numbers.
4. Unicode Support - Python 2 uses the ASCII alphabet by default, so when you
type "Hello, Sammy!" Python 2 will handle the string as ASCII.
Python 3 uses Unicode by default, which saves programmers extra development time,
and you can easily type and display many more characters directly into your program.
Because Unicode supports greater linguistic character diversity as well as the display of
emojis, using it as the default character encoding ensures that mobile devices around
the world are readily supported in your development projects.
5. Error Handling: There is a small change in error handling in both
versions. In python 3.x, ‘as’ keyword is required.

Define class?
Class is a specific object type created when class statement is executed.
How do we share global variable across modules in Python?
We can create a config file and store the entire global variable to be shared across
modules or scripts in it.
Describe how to implement cookies for web python?
A cookie is an arbitrary string of characters that uniquely identify a session. Each cookie
is specific to one website and one user.
What are uses of lambda?
It used to create small anonymous functions at run time.
How do I copy an object in Python?
Try copy.copy or copy.deepcopy for the general case. Not all objects can be copied, but
most can.

import copy
newobj = copy.copy(oldobj) # shallow copy
newobj = copy.deepcopy(oldobj) # deep (recursive) copy
Some objects can be copied more easily. Dictionaries have a copy method:

newdict = olddict.copy()
Sequences can be copied by slicing:

new_list = L[:]
You can also use the list, tuple, dict, and set functions to copy the corresponding objects,
and to convert between different sequence types:

new_list = list(L) # copy

new_dict = dict(olddict) # copy

new_set = set(L) # convert list to set

new_tuple = tuple(L) # convert list to tuple

How is memory managed in python?

Memory is managed in Python in following ways;
 Memory is managed in Python by private heap space. All Python objects and
data structures are located in a private heap.
 The programmer does not have access to this private heap and interpreter takes
care of this python private heap.
 Python memory manager is responsible for allocating python heap space for
Python objects.
 Python also have an inbuilt garbage collector, which recycle all the unused
memory and frees the memory and makes it available to heap space.
 The gc module defines functions to enable /disable garbage collector:
gc.enable() -Enables automatic garbage collection.
gc.disable() - Disables automatic garbage collection

What is namespace in python?

In python, every name has a place where it lives and can be hooked for. This is known
as namespace.
It is alike a box where a variable name is mapped to the object placed.
Whenever the variable is searched out, this box will be searched, to get corresponding
object.
Lists
What are extended slicing in Python?
L = [1,2,3,4,5]
Print(L[:])
Output: [1,2,3,4,5]
By passing a colon :, you will just copy the list!

It allows you to literally step through your list and select only those elements that your
step value includes.
L = range(10)
L[::2]
[0, 2, 4, 6, 8]

Negative values also work to make a copy of the same list in reverse order:
L[::-1]
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

This also works for tuples, arrays, and strings.

How to find size of a list?
List = [1,2,3,4,5]
Print(Len(list))
Output:5
Len()- function to find the length of list and tuple
How to find sum of elements in a list?
List = [1,2,3,4,5]
Print(sum(list))
Output: 15
sum() – function to find the sum of elements of a list

When To Use Python Lists And When To Use Tuples, Dictionaries Or Sets?
Sets
Unordered collection of unique objects.
set_one={1,2,3,4,5}
set_two = {4,5,6,7.8}
Functions:
 Add(): set_one.add(6)
 Union(): print(set_one.union(set_two))
 Intersection(): print(set_one.intersection(set_two))
 Difference(): print(set_one.difference(set_two))
Lists Versus Tuples
Tuples are used to collect an immutable ordered list of elements. This means that:
 You can’t add elements to a tuple. There’s no append() or extend() method
for tuples,
 You can’t remove elements from a tuple. Tuples have
no remove() or pop() method,
 You can find elements in a tuple since this doesn’t change the tuple.
 You can also use the in operator to check if an element exists in the tuple.
So, if you’re defining a constant set of values and all you’re going to do with it is iterate
through it, use a tuple instead of a list. It will be faster than working with lists and also
safer, as the tuples contain “write-protect” data.

Lists Versus Dictionaries

 A list stores an ordered collection of items, so it keeps some order. Dictionaries
don’t have any order.
 Dictionaries are known to associate each key with a value, while lists just contain
values.
Use a dictionary when you have an unordered set of unique keys that map to values.

Note that, because you have keys and values that link to each other, the performance
will be better than lists in cases where you’re checking membership of an element.

Lists Versus Sets

 Just like dictionaries, sets have no order in their collection of items. Not like lists.
 Set requires the items contained in it to be hashable, lists store non-hashable
items.
 Sets require your items to be unique and immutable. Duplicates are not allowed
in sets, while lists allow for duplicates and are mutable.
 List uses append() to add element but set uses add() to add element.
You should make use of sets when you have an unordered set of unique, immutable
values that are hashable.

How to select an element from a list?

We can select an element from a list by using an index operator [].

How to transform Python List to other Data structures?

List to a String
You convert a list to a string by using ''.join(). This operation allows you to glue together
all strings in your list together and return them as a string.
List to a Tuple
By using a tuple() function.
tuple(list)
List to set
The duplicate elements and order of the original list will be lost.
By using set() function.
set(list)
List to dictionary
zip() function returns a list of tuples, where the i-th tuple contains the i-th element from
each of the argument sequences or iterables. We need to pass this to dict() function.
What is the difference between Python append() and extend() method?
extend() - takes an iterable (it takes a list, set, tuple or string!), and adds each element
of the iterable to the list one at a time.
append() - adds its argument to the end of the list as a single item, which means that
when the append() function takes an iterable as its argument, it will treat it as a
single object.
How to concatenate two list?
With extend() and append() we modify the original list while with the + operator ,we can
make another list.
List3 = List1 + List2
How to sort a list in Python?
There are two ways:
 sort() method – used for sorting
 sorted() function - sorted() function can be applied to any Iterable object,
which means that it also accepts strings, sets, dictionaries when they are passed
to it.
Parameters:
o iterable
o reverse – if True, sorted list will be reversed.
o key – function that serves as a key for the sort comparision.
List2 = ["I", "am", "trying", "to", "learn", "python"]
L = sorted(List2, reverse = True, key = len) #len is an in-built function to count the length of an
element.
print(L)
Output: ['trying', 'python', 'learn', 'am', 'to', 'I']

How to clone or copy a list in Python?

 Slice original list and store it in new list
newlist = oldlist[:]
 Use built-in list() function
newlist = list(oldlist)
 Use copy() library
newlist = copy.copy(oldlist)
If the list contains object and we want to copy those as well :
newlist = copy.deepcopy(oldlist)
When you use the copy() methods, your original lists will be modified. However, if
you use the deepcopy() method, this will be prevented.
How to count occurrence of a list item in Python?
 count() = used to count the occurrences of just one item.
print([1, 2, 9, 4, 5, 4, 1].count(4))
Output: 2
 To count the occurrences of items in a list, you can also use list
comprehension in combination with the count()method:
list= ["a","b","b"]
[[x,list.count(x)] for x in set(list)]
Output: [['b', 2], ['a', 1]]
 Counter() method from collections library. It is a dict subclass to count
hashable objects. It is an unordered collection where elements are stored as
dictionary keys and their counts are stored as dictionary values. Counts are
allowed to be any integer value including zero or negative counts.
Print(Counter(list))
Output: Counter({'a': 1, 'b': 2})

How to create flat lists out of lists?

Using sum() function and pass our list and an empty list to it:
list = [[1,2],[3,4],[5,6]]
# Flatten out your original list of lists with `sum()`
print(sum(list, []))

How to find maximum and minimum of a list in Python?

Using max() and min() function and pass the list to it.
 Strings
What does ‘r’ before string literal do?
When prefixed with the letter ‘r’ or ‘R’ a string literal becomes a raw string and the
escape sequences such as \n are not converted.
print(r,"\nhello")
output: \nhello
str = 'hi it\'s me'
output: SyntaxError: invalid character in identifier
str = 'hi it\'s me'

What does max() returns in String?

Max returns the character with the highest ascii value.
example = "snow world"
print(max(example))
Output: w

How to count the occurrence of a character in a String?

Using count() function and passing the character as an argument.
String = “advghradca”
Print(String.count(“a”))
Output: 3

How to find index of a character in a String?

Using find() and rfind() function , passing character as an argument.
find() - returns lowest index .
rfind() – returns highest index.
example = "snow world"
print(example.find("o"))
print(example.rfind("o"))
output:2
output:6

How to replicate string?

Replication operator uses two parameter for operation. One is the integer value and
the other one is the String.
The Replication operator is used to repeat a string number of times. The string will
be repeated the number of times which is given by the integer value.
Eg:
>>> 5*"Vimal"

Output:
'VimalVimalVimalVimalVimal'

How to concatenate two strings?

‘+’ operator and “__add__” method is used for concatenation of two string.
S3 = S1 + S2
S3 = S1.__add__(S2)

What is the use of chr() and ord() in String?

chr(i)
Return the string representing a character whose Unicode code point is the integer i. For
example, chr(97) returns the string 'a', while chr(8364) returns the string '€'.
This is the inverse of ord().

The valid range for the argument is from 0 through 1,114,111 (0x10FFFF in base 16).
ValueError will be raised if i is outside that range.

ord(c)
Given a string representing one Unicode character, return an integer representing the
Unicode code point of that character. For example, ord('a') returns the
integer 97 and ord('€') (Euro sign) returns 8364. This is the inverse of chr().

How to replace character in a string?

S = “hello”
S.replace(“l”,e)
print(S)
Output: heeeo

How to get character at specified index in a string in Python?

__getitem(..) can be used to get character at index specified as parameter.
print(S.__ S = “hello”
getitem__(3))
output: l

What does capitalize() do in Python?

It returns a copy of the string with only its first character capitalized.
str.capitalize()

Str = “hello”
print(str.capitalize())
output: Hello

What does title() do in Python?

It is used to convert the first character in each word to Uppercase and
remaining characters to Lowercase in string and returns new string.
str1 = 'geeKs foR geEks'
str2 = str1.title()
print 'First Output after Title() method is = ', str2
Output:
First Output after Title() method is = Geeks For Geeks

What does split() does in Python?

split() method returns a list of strings after breaking the given string
by the specified separator.
Syntax :
str.split(separator, maxsplit)

del()-
sort()-To sort the list in ascending order.
count ()- This method counts the number of occurrences of one string within
another string. The simplest form is this one: s.count(substring). Only non-
overlapping occurrences are taken into account:

find()-The function returns the index of the first occurrence of the substring. If
the substring is not found, the method returns -1.

rfind()- returns the index of the last occurrence of the substring.

swapcase () - The method swapcase() returns a copy of the string in which all the case-
based characters have had their case swapped.
str.swapcase();
replace()- Method replace() replaces all occurrences of a given substring with
another one. Syntax: s.replace(old, new) takes the string S and replaces all
occurrences of substring old with the substring new.

What does center() does to string in Python?

The method center() returns centered in a string of length width. Padding is done using the specified
fillchar. Default filler is a space.
str.center(width[, fillchar])

Parameters

 width − This is the total width of the string.

  fillchar − This is the filler character.

How to check the start and end character of String in Python?

startswith()-The method startswith() checks whether string starts with str, optionally
restricting the matching with the given indices start and end.
str.startswith(str, beg=0,end=len(string));

endswith()- It returns True if the string ends with the specified suffix, otherwise return
False optionally restricting the matching with the given indices start and end.
str.endswith(suffix[, start[, end]])

How to find length of String in Python?

s.__len__()

Tuple
A tuple needn’t be enclosed in parenthesis.
a,b,c = 1,2,3
a,b,c
(1,2,3)

Tuple Unpacking: For unpacking to happen, equal number of values should be

there on both the sides.
a,b,c =1,2,3
ex: a,b = 1,2,3
too many values to unpack

Dictionary
The method setdefault() is similar to get(), but will set dict[key]=default if key
is not already in dict.
dict.setdefault(key, default=None)

Mention the differences between Django, Pyramid and Flask.

 Flask is a “microframework” primarily build for a small application with simpler
requirements. In flask, you have to use external libraries. Flask is ready to use.
  Pyramid is built for larger applications. It provides flexibility and lets the
developer use the right tools for their project. The developer can choose the
database, URL structure, templating style and more. Pyramid is heavy
configurable.
 Django can also used for larger applications just like Pyramid. It includes an
ORM.

How can you optimize memory usage in a Python application?

Some ways to optimize memory usage in a Python application:
 Use generators and iterators: Generators and iterators can help reduce memory usage by
allowing you to process data from one element simultaneously rather than loading the entire
dataset into memory at once.
 Use built-in functions and modules: Built-in functions and modules like "itertools" and
collections can help optimize memory usage by providing efficient algorithms and data
structures optimized for memory usage.
 Avoid unnecessary copying of data: Python objects are often passed by reference, which
can result in excessive data copying. To avoid this, you can use immutable objects like
tuples or copy() functions to create shallow rather than deep copies.
 Use lazy loading: Lazy loading is a technique in which data is loaded into memory only
when needed rather than the entire dataset. It can help reduce memory usage and improve
performance.

What is .pyc file in Python?

It is a python compile file created by the interpreter. It is used when we call the import statement in a
script.

To create a .pyc file manually:

import py_compile
py_compile.compile(“test.py”)
Output: __pycache__/test.cpython.36.pyc (name of .pyc file)
Itertools
Infinite Iterators:

Iterator Arguments Results Example

count() start, start+step, count(10) --> 10 11 12 13

start, [step] 14 ...
start+2*step, ...

cycle() p0, p1, ... plast, p0, cycle('ABCD') --> A B C D

p A B C D ...
p1, ...
elem, elem, elem, ...
repeat() elem [,n] endlessly or up to n repeat(10, 3) --> 10 10 10
times

itertools.count(start=0, step=1)
Make an iterator that returns evenly spaced values starting with n. Often used as an
argument to map() to generate consecutive data points. Also, used with zip() to add
sequence numbers. Equivalent to:
def count(start=0, step=1):
# count(10) --> 10 11 12 13 14 ...
# count(2.5, 0.5) -> 2.5 3.0 3.5 ...
n = start
while True:
yield n
n += step
When counting with floating point numbers, better accuracy can sometimes be achieved
by substituting multiplicative code such
as: (start + step * i for i in count()).

itertools.cycle(iterable)
Make an iterator returning elements from the iterable and saving a copy of each. When
the iterable is exhausted, return elements from the saved copy. Repeats indefinitely.
Equivalent to:
def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
saved = []
for element in iterable:
yield element
saved.append(element)
while saved:
for element in saved:
yield element
Note, this member of the toolkit may require significant auxiliary storage (depending on
the length of the iterable).

pytest
Pytest is a testing framework which allows us to write test codes using python.

Pytest expects our tests to be located in files whose names begin

with test_ or end with _test.py.
Advantages of Pytests
 Very easy to start with because of its simple and easy syntax.
 Can run tests in parallel.
 Can run a specific test or a subset of tests
 Automatically detect tests
 Skip tests
 Open source
 Numpy
Numpy is the core library for scientific computing in Python. It provides a high-performance
multidimensional array object, and tools for working with these arrays

Arrays
 A numpy array is a grid of values, all of the same type,
 is indexed by a tuple of nonnegative integers.
 The number of dimensions is the rank of the array;
 the shape of an array is a tuple of integers giving the size of the array along each
dimension.
We can initialize numpy arrays from nested Python lists, and access elements using square
brackets:
import numpy as np

a = np.array([1, 2, 3]) # Create a rank 1 array

print(type(a)) # Prints "<class 'numpy.ndarray'>"
print(a.shape) # Prints "(3,)"
print(a[0], a[1], a[2]) # Prints "1 2 3"
a[0] = 5 # Change an element of the array
print(a) # Prints "[5, 2, 3]"

b = np.array([[1,2,3],[4,5,6]]) # Create a rank 2 array

print(b.shape) # Prints "(2, 3)"
print(b[0, 0], b[0, 1], b[1, 0]) # Prints "1 2 4"

Numpy also provides many functions to create arrays:

import numpy as np

a = np.zeros((2,2)) # Create an array of all zeros

print(a) # Prints "[[ 0. 0.]
# [ 0. 0.]]"

b = np.ones((1,2)) # Create an array of all ones

print(b) # Prints "[[ 1. 1.]]"

c = np.full((2,2), 7) # Create a constant array

print(c) # Prints "[[ 7. 7.]
# [ 7. 7.]]"

d = np.eye(2) # Create a 2x2 identity matrix

print(d) # Prints "[[ 1. 0.]
# [ 0. 1.]]"
 e = np.random.random((2,2)) #Create an array filled with random
#values
print(e) # Might print "[[ 0.91940167 0.08143941]
# [ 0.68744134
0.87236687]

arange() will create arrays with regularly incrementing values. Check the docstring for complete
information on the various ways it can be used. A few examples will be given here:
>>> np.arange(10)

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> np.arange(2, 10, dtype=float)

array([ 2., 3., 4., 5., 6., 7., 8., 9.])

>>> np.arange(2, 3, 0.1)

array([ 2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9])

linspace() will create arrays with a specified number of elements, and spaced equally
between the specified beginning and end values. For example:

>>> np.linspace(1., 4., 6)

array([ 1. , 1.6, 2.2, 2.8, 3.4, 4. ])

The advantage of this creation function is that one can guarantee the number of elements
and the starting and end point, which arange() generally will not do for arbitrary start, stop,
and step values.

indices() will create a set of arrays (stacked as a one-higher dimensioned array), one per
dimension with each representing variation in that dimension. An example illustrates much
better than a verbal description:

>>> np.indices((3,3))

array([[[0, 0, 0], [1, 1, 1], [2, 2, 2]], [[0, 1, 2], [0, 1, 2], [0, 1, 2]]])

This is particularly useful for evaluating functions of multiple dimensions on a regular grid.

Some of the important attributes of a NumPy object are:

1. Ndim: displays the dimension of the array
2. Shape: returns a tuple of integers indicating the size of the array
3. Size: returns the total number of elements in the NumPy array
4. Dtype: returns the type of elements in the array, i.e., int64, character
5. Itemsize: returns the size in bytes of each item
6. Reshape: Reshapes the NumPy array
7. Resize: Returns a new array with the specified size.
One major difference is reshape() does not change your data but resize() does change
it. resize() first accommodates all the values in original array, after that if extra space is
there (or size of new array is greater than original array), then it adds its own values.
For resize() function, numpy.resize() returns a new copy of array
whereas ndarray.resize() does it in-place. But they don't go to view thing.
If the new array is larger than the original array, then the new array is filled with repeated
copies of a. Note that this behavior is different from a.resize(new_shape) which fills with
zeros instead of repeated copies of a.

Array indexing
Slicing:
Similar to Python lists, numpy arrays can be sliced. Since arrays may be multidimensional, you
must specify a slice for each dimension of the array:
import numpy as np

# Create the following rank 2 array with shape (3, 4)

# [[ 1 2 3 4]
# [ 5 6 7 8]
# [ 9 10 11 12]]
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])

# Use slicing to pull out the subarray consisting of the first 2

rows and columns 1 and 2; b is the following array of shape (2, 2):
# [[2 3]
# [6 7]]
b = a[:2, 1:3]

# A slice of an array is a view into the same data, so modifying it

# will modify the original array.
print(a[0, 1]) # Prints "2"
b[0, 0] = 77 # b[0, 0] is the same piece of data as a[0, 1]
print(a[0, 1]) # Prints "77"

Mix integer indexing with slice indexing.

import numpy as np

# Create the following rank 2 array with shape (3, 4)

# [[ 1 2 3 4]
# [ 5 6 7 8]
# [ 9 10 11 12]]
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
# Two ways of accessing the data in the middle row of the array.
# Mixing integer indexing with slices yields an array of lower
rank,
# while using only slices yields an array of the same rank as the
# original array:
row_r1 = a[1, :] # Rank 1 view of the second row of a
row_r2 = a[1:2, :] # Rank 2 view of the second row of a
print(row_r1, row_r1.shape) # Prints "[5 6 7 8] (4,)"
print(row_r2, row_r2.shape) # Prints "[[5 6 7 8]] (1, 4)"

# We can make the same distinction when accessing columns of an

array:
col_r1 = a[:, 1]
col_r2 = a[:, 1:2]
print(col_r1, col_r1.shape) # Prints "[ 2 6 10] (3,)"
print(col_r2, col_r2.shape) # Prints "[[ 2]
# [ 6]
# [10]] (3, 1)"

Integer array indexing:

When you index into numpy arrays using slicing, the resulting array view will always be a
subarray of the original array. In contrast, integer array indexing allows you to construct arbitrary
arrays using the data from another array.
import numpy as np

a = np.array([[1,2], [3, 4], [5, 6]])

# An example of integer array indexing.

# The returned array will have shape (3,) and
print(a[[0, 1, 2], [0, 1, 0]]) # Prints "[1 4 5]"

# The above example of integer array indexing is equivalent to

this:
print(np.array([a[0, 0], a[1, 1], a[2, 0]])) # Prints "[1 4 5]"

# When using integer array indexing, you can reuse the same
# element from the source array:
print(a[[0, 0], [1, 1]]) # Prints "[2 2]"

# Equivalent to the previous integer array indexing example

print(np.array([a[0, 1], a[0, 1]])) # Prints "[2 2]"

One useful trick with integer array indexing is selecting or mutating one element
from each row of a matrix:

import numpy as np
# Create a new array from which we will select elements
a = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])

print(a) # prints "array([[ 1, 2, 3],

# [ 4, 5, 6],
# [ 7, 8, 9],
# [10, 11, 12]])"

# Create an array of indices

b = np.array([0, 2, 0, 1])

# Select one element from each row of a using the indices in b

print(a[np.arange(4), b]) # Prints "[ 1 6 7 11]"

# Mutate one element from each row of a using the indices in b

a[np.arange(4), b] += 10

print(a) # prints "array([[11, 2, 3],

# [ 4, 5, 16],
# [17, 8, 9],
# [10, 21, 12]])

Boolean array indexing:

Boolean array indexing lets you pick out arbitrary elements of an array. Frequently this type of
indexing is used to select the elements of an array that satisfy some condition.
import numpy as np

a = np.array([[1,2], [3, 4], [5, 6]])

bool_idx = (a > 2) #Find the elements of a that are bigger than 2;

# this returns a numpy array of Booleans of
the same
# shape as a, where each slot of bool_idx
tells
# whether that element of a is > 2.

print(bool_idx) # Prints "[[False False]

# [ True True]
# [ True True]]"

# We use boolean array indexing to construct a rank 1 array

# consisting of the elements of a corresponding to the True values
# of bool_idx
print(a[bool_idx]) # Prints "[3 4 5 6]"

# We can do all of the above in a single concise statement:

print(a[a > 2]) # Prints "[3 4 5 6]"
Datatypes
Every numpy array is a grid of elements of the same type. Numpy provides a large set of
numeric datatypes that you can use to construct arrays. Numpy tries to guess a datatype when
you create an array, but functions that construct arrays usually also include an optional
argument to explicitly specify the datatype.
import numpy as np

x = np.array([1, 2]) # Let numpy choose the datatype

print(x.dtype) # Prints "int64"

x = np.array([1.0, 2.0]) # Let numpy choose the datatype

print(x.dtype) # Prints "float64"

x = np.array([1, 2], dtype=np.int64) # Force a particular

datatype
print(x.dtype) # Prints "int64"

Array math
Basic mathematical functions operate elementwise on arrays, and are available both as
operator overloads and as functions in the numpy module:
import numpy as np

x = np.array([[1,2],[3,4]], dtype=np.float64)
y = np.array([[5,6],[7,8]], dtype=np.float64)

# Elementwise sum; both produce the array

# [[ 6.0 8.0]
# [10.0 12.0]]
print(x + y)
print(np.add(x, y))

# Elementwise difference; both produce the array

# [[-4.0 -4.0]
# [-4.0 -4.0]]
print(x - y)
print(np.subtract(x, y))

# Elementwise product; both produce the array

# [[ 5.0 12.0]
# [21.0 32.0]]
print(x * y)
print(np.multiply(x, y))

# Elementwise division; both produce the array

# [[ 0.2 0.33333333]
# [ 0.42857143 0.5 ]]
print(x / y)
print(np.divide(x, y))

# Elementwise square root; produces the array

# [[ 1. 1.41421356]
# [ 1.73205081 2. ]]
print(np.sqrt(x))

Note that unlike MATLAB, * is elementwise multiplication, not matrix multiplication. We

instead use the dot function to compute inner products of vectors, to multiply a vector
by a matrix, and to multiply matrices. dot is available both as a function in the numpy
module and as an instance method of array objects:
import numpy as np

x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

v = np.array([9,10])
w = np.array([11, 12])

# Inner product of vectors; both produce 219

print(v.dot(w))
print(np.dot(v, w))

# Matrix / vector product; both produce the rank 1 array [29 67]
print(x.dot(v))
print(np.dot(x, v))

# Matrix / matrix product; both produce the rank 2 array

# [[19 22]
# [43 50]]
print(x.dot(y))
print(np.dot(x, y))

Numpy provides many useful functions for performing computations on arrays; one of
the most useful is sum :
import numpy as np

x = np.array([[1,2],[3,4]])

print(np.sum(x)) # Compute sum of all elements; prints "10"

print(np.sum(x, axis=0)) # Compute sum of each column; prints "[4
6]"
print(np.sum(x, axis=1)) # Compute sum of each row; prints "[3 7]"
Transposing Array
Apart from computing mathematical functions using arrays, we frequently need to
reshape or otherwise manipulate data in arrays. The simplest example of this type of
operation is transposing a matrix; to transpose a matrix, simply use the T attribute of an
array object:
import numpy as np

x = np.array([[1,2], [3,4]])
print(x) # Prints "[[1 2]
# [3 4]]"
print(x.T) # Prints "[[1 3]
# [2 4]]"

# Note that taking the transpose of a rank 1 array does nothing:

v = np.array([1,2,3])
print(v) # Prints "[1 2 3]"
print(v.T) # Prints "[1 2 3]"

Broadcasting
allows numpy to work with arrays of different shapes when performing arithmetic operations.
import numpy as np

# We will add the vector v to each row of the matrix x,

# storing the result in the matrix y
x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
v = np.array([1, 0, 1])
y = np.empty_like(x) # Create an empty matrix with the same shape
as x

# Add the vector v to each row of the matrix x with an explicit

loop
for i in range(4):
y[i, :] = x[i, :] + v

# Now y is the following

# [[ 2 2 4]
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]
print(y)

This works; however when the matrix x is very large, computing an explicit loop in Python
could be slow. Note that adding the vector v to each row of the matrix x is equivalent to
forming a matrix vv by stacking multiple copies of v vertically, then performing elementwise
summation of x and vv . We could implement this approach like this:
import numpy as np

# We will add the vector v to each row of the matrix x,

# storing the result in the matrix y
x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
v = np.array([1, 0, 1])
vv = np.tile(v, (4, 1)) # Stack 4 copies of v on top of each
other
print(vv) # Prints "[[1 0 1]
# [1 0 1]
# [1 0 1]
# [1 0 1]]"
y = x + vv # Add x and vv elementwise
print(y) # Prints "[[ 2 2 4
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]"

Numpy broadcasting allows us to perform this computation without actually creating

multiple copies of v . Consider this version, using broadcasting:

import numpy as np

# We will add the vector v to each row of the matrix x,

# storing the result in the matrix y
x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])
v = np.array([1, 0, 1])
y = x + v # Add v to each row of x using broadcasting
print(y) # Prints "[[ 2 2 4]
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]"
The line y = x + v works even though x has shape (4, 3) and v has shape (3,) due to
broadcasting; this line works as if v actually had shape (4, 3) , where each row was a copy
of v , and the sum was performed elementwise.

Broadcasting two arrays together follows these rules:

1. If the arrays do not have the same rank, prepend the shape of the lower
rank array with 1s until both shapes have the same length.
2. The two arrays are said to be compatible in a dimension if they have the
same size in the dimension, or if one of the arrays has size 1 in that dimension.
3. The arrays can be broadcast together if they are compatible in all
dimensions.
4. After broadcasting, each array behaves as if it had shape equal to the
elementwise maximum of shapes of the two input arrays.
 5. In any dimension where one array had size 1 and the other array had size
greater than 1, the first array behaves as if it were copied along that dimension

Here are some applications of broadcasting:

import numpy as np

# Compute outer product of vectors

v = np.array([1,2,3]) # v has shape (3,)
w = np.array([4,5]) # w has shape (2,)
# To compute an outer product, we first reshape v to be a column
# vector of shape (3, 1); we can then broadcast it against w to
yield
# an output of shape (3, 2), which is the outer product of v and w:
# [[ 4 5]
# [ 8 10]
# [12 15]]
print(np.reshape(v, (3, 1)) * w)

# Add a vector to each row of a matrix

x = np.array([[1,2,3], [4,5,6]])
# x has shape (2, 3) and v has shape (3,) so they broadcast to
# (2, 3), giving the following matrix:
# [[2 4 6]
# [5 7 9]]
print(x + v)

# Add a vector to each column of a matrix

# x has shape (2, 3) and w has shape (2,).
# If we transpose x then it has shape (3, 2) and can be broadcast
# against w to yield a result of shape (3, 2); transposing this
result
# yields the final result of shape (2, 3) which is the matrix x
with
# the vector w added to each column. Gives the following matrix:
# [[ 5 6 7]
# [ 9 10 11]]
print((x.T + w).T)
# Another solution is to reshape w to be a column vector of shape
(2, 1);
# we can then broadcast it directly against x to produce the same
# output.
print(x + np.reshape(w, (2, 1)))
# Multiply a matrix by a constant:
# x has shape (2, 3). Numpy treats scalars as arrays of shape ();
# these can be broadcast together to shape (2, 3), producing the
# following array:
# [[ 2 4 6]
# [ 8 10 12]]
print(x * 2)
 Pandas
Pandas is an open-source, BSD-licensed Python library providing high-
performance, easy-to-use data structures and data analysis tools for the Python
programming language.
Python with Pandas is used in a wide range of fields including academic and
commercial domains including finance, economics, Statistics, analytics, etc.
Using Pandas, we can accomplish five typical steps in the processing and
analysis of data, regardless of the origin of data — load, prepare, manipulate,
model, and analyze.
Python with Pandas is used in a wide range of fields including academic and
commercial domains including finance, economics, Statistics, analytics, etc.

Key Features of Pandas

 Fast and efficient DataFrame object with default and customized indexing.
 Tools for loading data into in-memory data objects from different file formats.
 Data alignment and integrated handling of missing data.
 Reshaping and pivoting of date sets.
 Label-based slicing, indexing and subsetting of large data sets.
 Columns from a data structure can be deleted or inserted.
 Group by data for aggregation and transformations.
 High performance merging and joining of data.
 Time Series functionality.

Pandas deals with the following three data structures −

 Series
 DataFrame
 Panel

Data Dimensions Description

Structure

Series 1 1D labeled homogeneous array, sizeimmutable.

Data Frames 2 General 2D labeled, size-mutable tabular structure

with potentially heterogeneously typed columns.

Panel 3 General 3D labeled, size-mutable array.

Mutability
All Pandas data structures are value mutable (can be changed) and except Series
all are size mutable.
Series
Series is a one-dimensional array like structure with homogeneous data.

10 23 56 17 52 61 73 90 26 72

Key Points
 Homogeneous data
 Size Immutable
 Values of Data Mutable
A pandas Series can be created using the following constructor −
pandas.Series( data, index, dtype, copy)

Empty Series
import pandas as pd
s = pd.Series()
print s

Create a Series from ndarray

import pandas as pd
import numpy as np
data = np.array(['a','b','c','d'])
s = pd.Series(data)
print s
We did not pass any index, so by default, it assigned the indexes ranging from 0
to len(data)-1, i.e., 0 to 3.

Output:
0 a
1 b
2 c
3 d
dtype: object

Example 2
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = np.array(['a','b','c','d'])
s = pd.Series(data,index=[100,101,102,103])
print s
Its output is as follows −
100 a
101 b
102 c
103 d
dtype: object

Create a Series from dict

A dict can be passed as input and if no index is specified, then the dictionary keys are taken in a
sorted order to construct index. If index is passed, the values in data corresponding to the labels in
the index will be pulled out.
Example 1
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = {'a' : 0., 'b' : 1., 'c' : 2.}
s = pd.Series(data)
print s
Its output is as follows −
a 0.0
b 1.0
c 2.0
dtype: float64
Observe − Dictionary keys are used to construct index.
Example 2
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = {'a' : 0., 'b' : 1., 'c' : 2.}
s = pd.Series(data,index=['b','c','d','a'])
print s
Its output is as follows −
b 1.0
c 2.0
d NaN
a 0.0
dtype: float64
Observe − Index order is persisted and the missing element is filled with
NaN (Not a Number).
Create a Series from Scalar
If data is a scalar value, an index must be provided. The value will be repeated
to match the length of index.
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
s = pd.Series(5, index=[0, 1, 2, 3])
print s
Its output is as follows −
0 5
1 5
2 5
3 5
dtype: int64

Accessing Data from Series with Position

Data in the series can be accessed similar to that in an ndarray.
Example 1
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve the first element

print s[0]
Its output is as follows −
1
Example 2
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve the first three element

print s[:3]
ts output is as follows −
a 1
b 2
c 3
dtype: int64

Retrieve Data Using Label (Index)

A Series is like a fixed-size dict in that you can get and set values by index
label.
Example 1

Retrieve a single element using index label value.

import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve a single element

print s['a']
Its output is as follows −
1

Example 2

Retrieve multiple elements using a list of index label values.

import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve multiple elements

print s[['a','c','d']]
Its output is as follows −
a 1
c 3
d 4
dtype: int64
Example 3

If a label is not contained, an exception is raised.

import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve multiple elements

print s['f']
Its output is as follows −
…
KeyError: 'f'

DataFrame
DataFrame is a two-dimensional array with heterogeneous data. For example,

Name Age Gender Rating

Steve 32 Male 3.45

Lia 28 Female 4.6

Vin 45 Male 3.9

Katie 38 Female 2.78

Key Points
 Heterogeneous data
 Size Mutable
 Data Mutable
  Labeled axes (rows and columns)
 Can Perform Arithmetic operations on rows and columns
pandas.DataFrame
pandas.DataFrame( data, index, columns, dtype, copy)

Create DataFrame
A pandas DataFrame can be created using various inputs like −
 Lists
 dict
 Series
 Numpy ndarrays
 Another DataFrame

Create an Empty DataFrame

#import the pandas library and aliasing as pd
import pandas as pd
df = pd.DataFrame()
print df

Its output is as follows −

Empty DataFrame
Columns: []
Index: []
Create a DataFrame from Lists
The DataFrame can be created using a single list or a list of lists.
Example 1
import pandas as pd
data = [1,2,3,4,5]
df = pd.DataFrame(data)
print df
Its output is as follows −
0
0 1
1 2
2 3
3 4
4 5
Example 2
import pandas as pd
data = [['Alex',10],['Bob',12],['Clarke',13]]
df = pd.DataFrame(data,columns=['Name','Age'])
print df
Its output is as follows −
Name Age
0 Alex 10
1 Bob 12
2 Clarke 13
Example 3
import pandas as pd
data = [['Alex',10],['Bob',12],['Clarke',13]]
df = pd.DataFrame(data,columns=['Name','Age'],dtype=float)
print df
Its output is as follows −
Name Age
0 Alex 10.0
1 Bob 12.0
2 Clarke 13.0
Note − Observe, the dtype parameter changes the type of Age column to
floating point.
Create a DataFrame from Dict of ndarrays / Lists
All the ndarrays must be of same length. If index is passed, then the
length of the index should equal to the length of the arrays.

If no index is passed, then by default, index will be range(n), where n is the

array length.
Example 1
import pandas as pd
data = {'Name':['Tom', 'Jack', 'Steve', 'Ricky'],'Age':[28,34,29,42]}
df = pd.DataFrame(data)
print df
Its output is as follows −
Age Name
0 28 Tom
1 34 Jack
2 29 Steve
3 42 Ricky
Note − Observe the values 0,1,2,3. They are the default index assigned to
each using the function range(n).
Example 2

Let us now create an indexed DataFrame using arrays.

import pandas as pd
data = {'Name':['Tom', 'Jack', 'Steve', 'Ricky'],'Age':[28,34,29,42]}
df = pd.DataFrame(data, index=['rank1','rank2','rank3','rank4'])
print df
Its output is as follows −
Age Name
rank1 28 Tom
rank2 34 Jack
rank3 29 Steve
rank4 42 Ricky
Note − Observe, the index parameter assigns an index to each row.

Create a DataFrame from List of Dicts

List of Dictionaries can be passed as input data to create a DataFrame. The
dictionary keys are by default taken as column names.
Example 1

The following example shows how to create a DataFrame by passing a list of

dictionaries.
import pandas as pd
data = [{'a': 1, 'b': 2},{'a': 5, 'b': 10, 'c': 20}]
df = pd.DataFrame(data)
print df
Its output is as follows −
a b c
0 1 2 NaN
1 5 10 20.0
Note − Observe, NaN (Not a Number) is appended in missing areas.
Example 2

The following example shows how to create a DataFrame by passing a list of

dictionaries and the row indices.
import pandas as pd
data = [{'a': 1, 'b': 2},{'a': 5, 'b': 10, 'c': 20}]
df = pd.DataFrame(data, index=['first', 'second'])
print df
Its output is as follows −
a b c
first 1 2 NaN
second 5 10 20.0
Example 3

The following example shows how to create a DataFrame with a list of

dictionaries, row indices, and column indices.
import pandas as pd
data = [{'a': 1, 'b': 2},{'a': 5, 'b': 10, 'c': 20}]

#With two column indices, values same as dictionary keys

df1 = pd.DataFrame(data, index=['first', 'second'], columns=['a', 'b'])

#With two column indices with one index with other name
df2 = pd.DataFrame(data, index=['first', 'second'], columns=['a', 'b1'])
print df1
print df2
Its output is as follows −
#df1 output
a b
first 1 2
second 5 10

#df2 output
a b1
first 1 NaN
second 5 NaN
Note − Observe, df2 DataFrame is created with a column index other than the dictionary key;
thus, appended the NaN’s in place. Whereas, df1 is created with column indices same as
dictionary keys, so NaN’s appended.

Panel
Panel is a three-dimensional data structure with heterogeneous data. It is hard
to represent the panel in graphical representation. But a panel can be illustrated as
a container of DataFrame.

Key Points
 Heterogeneous data
 Size Mutable
Data Mutable

Pandas DataFrame Analysis

Pandas DataFrame objects come with a variety of built-in functions
like head() , tail() and info() that allow us to view and analyze DataFrames.

Pandas head()
The head() method provides a rapid summary of a DataFrame. It returns the column
headers and a specified number of rows from the beginning.
import pandas as pd

# create a dataframe
data = {'Name': ['John', 'Alice', 'Bob', 'Emma', 'Mike', 'Sarah', 'David', 'Linda',
'Tom', 'Emily'],
'Age': [25, 30, 35, 28, 32, 27, 40, 33, 29, 31],
'City': ['New York', 'Paris', 'London', 'Sydney', 'Tokyo', 'Berlin', 'Rome',
'Madrid', 'Toronto', 'Moscow']}
df = pd.DataFrame(data)
# display the first three rows
print('First Three Rows:')
print(df.head(3))
print()

# display the first five rows

print('First Five Rows:')
print(df.head())

Output

First Three Rows:

Name Age City
0 John 25 New York
1 Alice 30 Paris
2 Bob 35 London

First Five Rows:

Name Age City
0 John 25 New York
1 Alice 30 Paris
2 Bob 35 London
3 Emma 28 Sydney
4 Mike 32 Tokyo

In this example, we displayed selected rows of the df DataFrame starting from the top using
head().

Notice that the first five rows are selected by default when no argument is passed to the head()
method.

Pandas tail()
The tail() method is similar to head() but it returns data starting from the end
of the DataFrame.

Full Stack Web Development with Django and Python

Two Important component:
 Front-end
HTML, CSS, JS
Jquery, bootstrap