Data Manipulation (Part I)

Week 4
National College of Ireland
Dublin, Ireland.

Lecturer: Dr. Muhammad Iqbal

Email: [email protected] 1
 Outline
• Purpose and importance of SQL.
• How to write an SQL command.
• How to retrieve data from database using SELECT and:
– Use simple and compound WHERE conditions.
– Sort query results using ORDER BY.
– Use AGGREGATE functions.
– Group data using GROUP BY and HAVING.
– Use subqueries.
– Join tables together.
– Perform set operations (UNION, INTERSECT, EXCEPT).
• How to update database using INSERT, UPDATE, and DELETE.
2
 History of SQL
• 1970–E. F. Codd has given the idea of a relational database

• 1974-1979 – System R with Sequel (later SQL) created at IBM

Research Lab.

• SQL first became an ANSI standard in 1986 and an ISO standard in

1987.

• SQL is by far the most important standard within the modern-day

relational database market.

• Practical systems, such as IBM DB2 or Oracle, will typically provide a

much richer set of SQL commands designed to manage and optimise
3
that particular system.
 Objectives of SQL
• Ideally, database language should allow user to:
1. Create the database and relation structures
2. Perform insertion, modification, deletion of data from
the relations
3. Perform simple and complex queries

• A database language must perform these tasks with minimal

user effort, and its command structure and syntax must be
relatively easy to learn.

• It must be portable (capable of being used on different

computer systems). 4
 Objectives of SQL

• SQL is relatively easy to learn:

– it is non-procedural

– we can specify what information requires for a

specific task, rather than how to get it

– it is essentially free-format

5
 Objectives of SQL
• SQL is a transform-oriented language with two major components:

i. A DDL (Data Definition Language) for defining database structure. For

example, CREATE tables, indexes, views, DROP OR ALTER tables.

ii. A DML (Data Manipulation Language) for retrieving and updating data.
For example, INSERT, UPDATE and SELECT Statements.

iii. A DCL (Data Control Language) to control access to data stored in a

database (Authorization). For example, GRANT, ADD, REVOKE.

• Until SQL: 1999, SQL did not contain flow of control commands. These
had to be implemented using a programming or job-control language,
or interactively by the decisions of user. 6
 Objectives of SQL
mysql> CREATE Schema/ Database TestingSQL
• Consists of standard English words:
1) CREATE TABLE Staff ( Sno VARCHAR(5),
fName VARCHAR(15),
lName VARCHAR(15),
StartDate DATE,
salary DECIMAL(7,2),
Primary Key (Sno));
2) INSERT INTO Staff VALUES ('SG16', ‘Peter’, 'Brown', ‘2000-01-17’, 8300);
3) SELECT Sno, lName, salary FROM Staff
7
WHERE salary > 10000;
 Datatypes in SQL
• MySQL supports a number of SQL
standard data types in various
categories. Following Data types are
based on MySQL community server
5.6.

LONGBLOB

1. String

2. Numeric

3. DATETIME

8
 Date
Datatypes

• The exact numeric data type is used to define numbers with an exact
representation.

• An exact numeric data type consists of a precision and a scale.

• The precision gives the total number of significant decimal digits; that
is, the total number of digits, including decimal places but excluding
the point itself.

• The scale gives the total number of decimal places.

• Example, the exact numeric value −12.345 has precision 5 and scale 3.

• We can store images in MySQL using the data type blob, however, it is
not recommended to do this. 9
 Writing SQL Commands
SQL statement consists of reserved words and user-defined words.

– Reserved words are a fixed part of SQL and must be spelt exactly
as required and cannot be split across lines.

– User-defined words are made up by user and represent names of

various database objects such as relations, columns and views.

• Most components of an SQL statement are case insensitive, except for literal
character data. It is more readable with indentation and lineation:

– Each clause should begin on a new line.

– Start of a clause should line up with start of other clauses.

– If the clause has several parts, should each appear on a separate line and
be indented under start of clause. 10
 Writing SQL Commands
• Use extended form of Backus Naur Form (BNF) notation:
– Upper-case letters represent reserved words.
– Lower-case letters represent user-defined words.
– | indicates a choice among alternatives.
– Curly braces ({ }) indicate a required element.
– Square brackets ([ ]) indicate an optional element.
– … indicates optional repetition (0 or more).

• Literals are constants used in SQL statements.

• All non-numeric literals must be enclosed in single quotes (e.g. 'London').

• All numeric literals must not be enclosed in quotes (e.g. 650.00). 11

 SELECT Statement
SELECT [DISTINCT | ALL] Optional Component
{* | [columnExpression [AS newName]] [,...] }
FROM TableName [alias] [, ...]
Required Element
[WHERE condition]
[GROUP BY columnList]
SELECT Specifies which columns are to
[HAVING condition]
[ORDER BY columnList] appear in the output.
FROM Specifies table(s) to be used.
WHERE Filters rows (Checking condition).
• Order of the clauses cannot GROUP BY Forms groups of rows with same
be changed.
column value.
• Only SELECT and FROM
HAVING Filters groups subject to some
are mandatory.
condition.
ORDER BY Specifies the order of the output.12
 All and Specific Columns
List full details of all staff.
SELECT Sno, fName, lName,
address, position, sex, DOB, salary, Bno
FROM Staff;
• Can also use * as an abbreviation
for 'all columns':
SELECT * Specify an asterisk (*)
FROM Staff;

Produce a list of salaries for all

staff, showing only = staff number,
first and last names, and salary.
SELECT Sno, fName, lName, salary
FROM Staff; 13
 Use of DISTINCT

List the property numbers of all

properties that have been viewed.
SELECT Pno
FROM Viewing;

• Use DISTINCT to eliminate

duplicates:
SELECT DISTINCT Pno
FROM Viewing; 14
 Calculation on Data Fields

Produce a list of monthly salaries for all staff, showing

staff number, first/last name, and salary.
SELECT Sno, fName, lName, salary/12
FROM Staff;

• To name column, use AS salary/12

clause:

SELECT Sno, fName, lName,

salary/12 AS monthlySalary
FROM Staff; 15
 Row selection (WHERE clause)
• Restriction of rows is possible with the WHERE clause, which consists of the
keyword WHERE followed by a search condition that specifies the rows to be
retrieved. For example,

1. Comparison: Compare the value of one expression to the value of another

expression. (Single and multiple values/ points of comparison)

2. Range: Test whether the value of an expression falls within a specified

range of values. (called as Range query)

3. Set Membership: Test whether the value of an expression equals one of a

set of values.

4. Pattern Match: Test whether a string matches a specified pattern. (Pre

and post fix match)
16
5. Null: Test whether a column has a null (unknown) value.
 1. Comparison Search Condition

List all staff with a salary greater than 10,000.

SELECT Sno, fName, lName, position, salary
FROM Staff
WHERE salary > 10000;

Logical comparison operators

=, <, <=, >, >=, and <> 17
1. Compound Comparison Search
Condition
List addresses of all branch offices in London
or Glasgow.
SELECT *
FROM Branch
WHERE city = 'London' OR city = 'Glasgow';

18
 2. Range Search Condition
List all staff with a salary between 20,000 and 30,000.
SELECT Sno, fName, lName, position, salary
FROM Staff
WHERE salary BETWEEN 20000 AND 30000;

• BETWEEN test includes the endpoints of range.

19
 2. Range Search Condition

• Also a negated version NOT BETWEEN.

• BETWEEN does not add much to SQL's expressive
power. Could also write:
SELECT Sno, fName, lName, position, salary
Comparative operators
FROM Staff
WHERE salary >= 20000 AND salary <= 30000;
• Useful, though, for a range of values.
i. = equal to, > greater than
ii. < less than , >= greater than
20
iii. <= less than or equal to, <> not equal to
 3. Set Membership

List all managers and supervisors.

SELECT Sno, fName, lName, position
FROM Staff
WHERE position IN ('Manager', 'Supervisor');

21
 3. Set Membership

• There is a negated version (NOT IN).

• IN does not add much to SQL's expressive power and
can be expressed this as:

SELECT Sno, fName, lName, position

FROM Staff
WHERE position = 'Manager' OR
position = 'Supervisor';

• IN is more efficient when set contains many values. 22

 4. Pattern Matching

Find all owners with the string 'Glasgow' in their

address.
SELECT Ono, fName, lName, address, Tel_No
FROM Owner
WHERE address LIKE '%Glasgow%';

• The LIKE keyword used in a WHERE operator with a wildcard (% or _)

23
allows you to search for patterns in character-based fields.
 4. Pattern Matching
• SQL has two special pattern matching symbols:
– % replaces an arbitrary number of zero or more characters
– underscore (_) replaces a single character
• LIKE '%Glasgow%' means a sequence of characters of any length containing
'Glasgow’. For example:

• Prefix match query means that the data starting with a particular value, (“Ad%”)

• Postfix match query means that the data ending with a particular value (“%lly”)24
 4. Pattern Matching
• For example: Address LIKE 'H%' means the first character
must be H, but the rest of the string can be anything.

• Address LIKE 'H_ _ _' means that there must be exactly four
characters in the string, the first of which must be an H.

• Address LIKE '%e' means any sequence of characters, of length

at least 1, with the last character an ‘e’.

• Address LIKE '%Glasgow%' means a sequence of characters of

any length containing Glasgow.

• Address NOT LIKE 'H%' means first character cannot be an25H.

 5. NULL Search Condition

List details of all viewings on property PG04 where a

comment has not been supplied.
• There are 2 viewings for property PG04, one with and
one without a comment.
• Have to test for null explicitly using special keyword
IS NULL:
Rno Date
SELECT Rno, Date
FROM Viewing
WHERE Pno = 'PG04' AND comment IS NULL; 26
 Single Column Ordering

List salaries for all staff, arranged

in descending order of salary.
SELECT Sno, fName, lName, salary
FROM Staff
ORDER BY salary DESC;
Keyword DESC to see result in a
descending order of values Keyword ASC
Produce abbreviated list of to specify ascending order explicitly

properties in order of property

type.
SELECT Pno, type, rooms, rent
FROM Property_For_Rent
ORDER BY type; 27
 SELECT Statement - Aggregates

ISO standard defines five aggregate functions:

1. COUNT returns number of values in a specified column.

2. SUM returns sum of values in a specified column.

3. AVG returns average of values in a specified column.

4. MIN returns smallest value in a specified column.

5. MAX returns largest value in a specified column.

28
 SELECT Statement - Aggregates
• Each operates on a single column of a table and returns a single value.

• COUNT, MIN, and MAX apply to numeric and non-numeric fields,

but SUM and AVG may be used on numeric fields only.

• Apart from COUNT(*), each function eliminates nulls first and

operates only on remaining non-null values.

• COUNT(*) counts all rows of a table, regardless of whether nulls or

duplicate values occur.

• Can use DISTINCT before column name to eliminate duplicates.

• DISTINCT has no effect with MIN/ MAX, but may have with SUM/
AVG. 29
 SELECT Statement - Aggregates

• Aggregate functions can be used only in SELECT list and in

HAVING clause.

• If SELECT list includes an aggregate function and there is no

GROUP BY clause, SELECT list cannot reference a column out
with an aggregate function. For example, the following is illegal:

SELECT Sno, COUNT(salary)

FROM Staff;

SELECT position, COUNT(salary)

FROM Staff
Group by position; 30
 Use of COUNT(*)

How many properties cost more than £350 per month to

rent?
SELECT COUNT(*) AS myCount
FROM Property_For_Rent
WHERE rent > 350;

31
 Use of COUNT(DISTINCT)

How many different properties viewed in May 13 1995?

SELECT COUNT(DISTINCT Pno) AS myCount
FROM Viewing
WHERE Date
BETWEEN '1995-04-01' AND '1995-04-31';

Format of Date is very

important in comparison
help DATE 32
 Use of COUNT and SUM

Find number of Managers and sum of their salaries.

SELECT COUNT(Sno) AS myCount,
SUM(salary) AS mySum
FROM Staff
Aggregate Functions
WHERE position = 'Manager';

33
http://dev.mysql.com/doc/refman/5.7/e
n/group-by-functions.html
 Use of MIN, MAX, AVG

Find minimum, maximum, and average staff

salary.
SELECT MIN(salary) AS myMin,
MAX(salary) AS myMax,
AVG(salary) AS myAvg
FROM Staff;

34
 SELECT Statement - Grouping

• Use GROUP BY clause to get sub-totals.

• SELECT and GROUP BY closely integrated: each item

in the SELECT list must be single-valued per group, and
SELECT clause may only contain:
– Column names
– Aggregate functions
– Constants
– Expression involving combinations of the above 35
 SELECT Statement - Grouping

• All column names in the SELECT list must appear in

GROUP BY clause unless name is used only in an
aggregate function.

• If WHERE is used with GROUP BY, WHERE is

applied first, then groups are formed from remaining
rows satisfying predicate.

• ISO considers two nulls to be equal for purposes of

GROUP BY.
36
 Use of GROUP BY

Find number of staff in each branch and their total

salaries.
SELECT Bno,
COUNT(Sno) AS myCount,
SUM(salary) AS mySum
FROM Staff
GROUP BY Bno
ORDER BY Bno;

37
 Restricted Groupings
HAVING clause
• HAVING clause is designed for use with GROUP
BY to restrict groups that appear in the final result
table.

• Similar to WHERE, but WHERE filters individual

rows whereas HAVING filters groups.

• Column names in HAVING clause must also

appear in the GROUP BY list or be contained
within an aggregate function.
38
 Use of HAVING
For each branch with more than 1 member of staff,
find number of staff in each branch and sum of their
salaries.
SELECT Bno,
COUNT(Sno) AS myCount,
SUM(salary) AS mySum
FROM Staff
GROUP BY Bno
HAVING COUNT(Sno) > 1
ORDER BY Bno; 39
 Subqueries

• Some SQL statements can have a SELECT embedded

within them.

• A subselect can be used in WHERE and HAVING

clauses of an outer SELECT, where it is called a
subquery or nested query.

• Subselects may also appear in INSERT, UPDATE, and

DELETE statements.
40
 Subquery with Equality

List staff who work in branch at '163 Main St'.

Outer SELECT

SELECT Sno, fName, lName, position

FROM Staff
WHERE Bno =
(SELECT Bno
FROM Branch
WHERE street = '163 Main St');
Inner SELECT 41
 Subquery with Equality

• Inner SELECT finds branch number for branch at

'163 Main St' ('B3').
• Outer SELECT then retrieves details of all staff who
work at this branch.
• Outer SELECT then becomes:
SELECT Sno, fName, lName, position
FROM Staff
WHERE Bno = 'B3';
42
 Subquery with Aggregate

List all staff whose salary is greater than the average salary, and show by
how much.
SELECT Sno, fName, lName, position,
Salary - (SELECT AVG(salary) FROM Staff) As SalDiff
FROM Staff
WHERE salary >
(SELECT AVG(salary)
FROM Staff);

43
 Subquery with Aggregate

• Cannot write 'WHERE salary > AVG(salary)'

• Instead, use subquery to find average salary (17000), and then use
outer SELECT to find those staff with salary greater than this:

SELECT Sno, fName, lName, position,

salary - 17000 As salDiff
FROM Staff
WHERE salary > 17000;

44
Review Questions

45
 Module References/ Resources
Recommended Book Resources Dr. Muhammad M Iqbal*
• Thomas Connolly, Carolyn Begg 2014, Database Systems: A Practical Approach
to Design, Implementation, and Management, 6th Edition Ed., Pearson
Education [ISBN: 1292061189] [Present in our Library]
Supplementary Book Resources
• Gordon S. Linoff, Data Analysis Using SQL and Excel, Wiley [ISBN:
0470099518]
• Eric Redmond, Jim Wilson, Seven Databases in Seven Weeks, Pragmatic
Bookshelf [ISBN: 1934356921]
• Baron Schwartz, Peter Zaitsev, Vadim Tkachenko, High Performance MySQL,
O'Reilly Media [ISBN: 1449314287]
Other Resources
• Website: http://www.thearling.com
• Website: http://www.mongodb.org
46
• Website: http://www.mysql.com