In The United States District Court Western District of Texas Waco Division

Case 6:21-cv-00663-ADA Document 1 Filed 06/25/21 Page 1 of 29
IN THE UNITED STATES DISTRICT COURT

WESTERN DISTRICT OF TEXAS
WACO DIVISION
INGENIOSHARE, LLC,
Plaintiff,
Civil Action No. 6:21-cv-00663
v.
EPIC GAMES, INC., JURY TRIAL DEMANDED
Defendant.
COMPLAINT FOR PATENT INFRINGEMENT
Plaintiff IngenioShare, LLC (“Plaintiff”) states for its Complaint against Epic
Games, Inc. as follows:
INTRODUCTION
1. This is an action for patent infringement arising under the patent laws of
the United States, Title 35, United States Code.
PARTIES
2. Plaintiff IngenioShare, LLC is a limited liability company organized
and existing under laws of California.
3. On information and belief, Defendant Epic Games, Inc. (“Defendant”)
is a corporation organized under the laws of Maryland, having a place of business
1
in Austin, Texas. Epic Games is registered to do business in Texas and can be served
in Texas through its registered agent, C T Corporation System, 1999 Bryan St., Ste.
900, Dallas, TX 75201.
4. Defendant does business in Texas, directly or through intermediaries
and offers products or services, including those accused herein of infringement, to
customers and potential customers located in Texas, including in this Judicial
District.
JURISDICTION
5. This Court has subject matter jurisdiction over all causes of action set
forth herein pursuant to 28 U.S.C. §§ 1331 and 1338(a) because this action arises
under the patent laws of the United States, Title 35, United States Code, including 35
U.S.C. § 271, et seq.
6. Defendant is subject to personal jurisdiction in the State of Texas and
in this Judicial District.
7. Venue for this suit is proper in this Judicial District pursuant to 28
U.S.C. § 1400(b). Defendant is registered to do business in Texas and maintains an
agent for service of process in Texas. On information and belief, Defendant
maintains a place of business within the Western District of Texas, as evidenced by
2
its website advertising job positions in Austin, Texas. On information and belief,
Defendant has also committed acts of infringement in this Judicial District.
8. On information and belief, Defendant has sold and offered for sale
accused products to residents of this Judicial District.
9. Defendant also has authorized retailers that offer and sell accused
products on its behalf in this Judicial District. These include, for example, Best
Buy stores, including those located at, for example, 4627 S Jack Kultgen Expy,
Waco, TX 76706; 3550 S General Bruce Dr., Temple, TX 76504; 3209 E. Central
Texas Expy., Killeen, TX 76543; 125 NW Loop 410, San Antonio, TX 78216;
6001 NW Loop 410, San Antonio, TX 78238; 2003 N Loop 1604 E, San Antonio,
TX 78232; 12909 Shops Pkwy, Bee Cave, TX 78738; and 9600 S Interstate 35,
Austin, TX 78748.
10. Plaintiff’s cause of action arises directly from Defendant’s business
contacts and other activities in the State of Texas and this Judicial District.
11. On information and belief, Defendant has derived substantial revenues
from its infringing acts occurring within the State of Texas and within this Judicial
District.
3
12. This Court has personal jurisdiction over Defendant at least because it
maintains an established place of business in this Judicial District. Defendant also has
employees working in and from this Judicial District.
13. This Court also has personal jurisdiction over Defendant under the
provisions of the Texas Long Arm Statute and consistent with Constitutional due
process by virtue of the fact that, upon information and belief, Defendant has
availed itself of the privilege of conducting and soliciting business within this
State, including engaging in at least some of the infringing activities in this State, as
well as by others acting as Defendant’s agents and/or representatives, such that it
would be reasonable for this Court to exercise jurisdiction consistent with
principles underlying the U.S. Constitution and without offending traditional
notions of fair play and substantial justice.
14. On information and belief, Defendant has also established minimum
contacts with this Judicial District and regularly transacts and does business within
this Judicial District, including advertising, promoting and selling products over
the Internet, through intermediaries, representatives and/or agents located within
this Judicial District, that infringe Plaintiff’s patents, which products are then sold
and/or shipped directly to citizens residing within this State and in this Judicial
District. Upon further information and belief, Defendant has purposefully directed
4
activities at citizens of this State, including those located within this Judicial
District.
15. On information and belief, Defendant has also purposefully and
voluntarily placed its products into the stream of commerce with the expectation
that they will be purchased and used by customers located in the State of Texas.
On information and belief, Defendant’s customers in the State of Texas, including
those in this Judicial District, have purchased and used and continue to purchase
and use Defendant’s products.
16. Furthermore, this Court has personal jurisdiction over Defendant under
the Long Arm Statute of the State of Texas because: (i) Defendant has and continues
to intentionally sell products and methods to customers in Texas; (ii) Defendant has
and continues to intentionally instruct customers and potential customers in Texas
with respect to how to use the products and methods that Defendant sells to
customers in Texas; (iii) Defendant knows and has known its products and methods,
including the infringing products and methods, have and continue to be sold and
marketed in Texas through regular and established distribution channels; (iv)
Defendant knows and has known its products and methods will enter and/or be
distributed throughout the United States of America and the State of Texas; (v)
Defendant has and continues to target customers and potential customers in Texas to
5
buy and/or use Defendant’s products and methods, including the infringing products
and methods; (vi) Defendant has and continues to provide advice to customers in
Texas; (vii) it has been and continues to be foreseeable that Defendant’s products and
methods, including the infringing products and methods, would enter the State of
Texas; (viii) Defendant has and continues to market to citizens of Texas through its
website; (ix) Defendant has and continues to provide services to citizens of Texas
through its website; (x) Defendant derives substantial revenue from Texas; (xi) Texas
has and continues to be part of Defendant’s established distribution channels; (xii)
the assertion of personal jurisdiction over Defendant is reasonable and fair; and (xiii)
the State of Texas has an interest in this matter due to the presence of Defendant
within this State, as well as the presence of Defendant’s infringing products in the
State of Texas.
17. This Court also has personal jurisdiction over Defendant because: (i)
Defendant maintains regular and systematic business contacts with the State of Texas
and within this Judicial District; (ii) Defendant purposely, regularly, and
continuously conducts business in the State of Texas and within this Judicial District;
(iii) Defendant purposefully directs its activities at residents of the State of Texas;
(iv) the causes of action set forth herein arise out of or relate to Defendant’s activities
in the State of Texas; and (v) the exercise of jurisdiction over Defendant will not
6
offend the traditional notions of fair play and substantial justice.
Factual Background
18. Plaintiff has partnered with inventors and entrepreneurs to provide
innovative market-driven ideas and user-friendly technologies. One set of its novel
solutions relates to communication technologies, applicable to multiplayer video
games.
19. Plaintiff’s predecessor-in-interest partnered with Kwok-wai Cheung,
Ph.D. in developing this innovative technology. Dr. Cheung received his B.S. (Eng.)
from the University of Hong Kong (1st Class Honor), followed by an M.S. from Yale
University and a Ph.D. from the California Institute of Technology.
20. Dr. Cheung was a member of the technical staff in the Applied Research
Area of Bell Communications Research from 1987 until 1992.
21. Since 1996, Dr. Cheung has been a professor in the Department of
Information Engineering at the Chinese University of Hong Kong. He served as the
Division Head of the Department of Information Engineering from Aug. 1, 2009
through July 31, 2012. He also served as the Director of the Centre for Innovation
and Technologies, Faculty of Engineering, at the Chinese University of Hong Kong
from May 1996 until September 2001.
22. Dr. Cheung is a named inventor on more than 40 patents, including
7
more than 20 United States patents.
23. Dr. Cheung has contributed to over 130 papers with over 800 Science
Citation Index (SCI) citations.
24. Dr. Cheung has been the principal investigator or co-principal
investigator on research grants totaling more than $10,000,000 (U.S.).
25. Dr. Cheung is the first-named inventor on the patents asserted in this
case.
26. Dr. Cheung, as well as the other named inventors, remain associated
with Plaintiff.
27. Defendant is a successful video game developer and publisher. In
2017, Defendant first released Fortnite, which is an online video game. Normally,
Fortnite is played by multiple players concurrently. On information and belief,
Fortnite has at least three distinct game mode versions that have the same general
gameplay and game engine: Fortnite: Save the World, which is a cooperative
hybrid-tower, defense-shooter-survival game in which four players fight
off zombie-like creatures and defend objects with traps and fortifications they
might build; Fortnite Battle Royale, a battle royale game in which up to 100
players fight to be the last person standing; and Fortnite Creative, in which players
are given complete freedom to create worlds and battle arenas.
8
28. Fortnite allows players to communicate with one another. Such
communication features are crucial to Fortnite. As explained by Defendant’s CEO
Tim Sweeney, part of the appeal of Fortnite is that it is “as much a social hangout
for a lot of folks as it is a game.” https://www.wsj.com/articles/the-man-behind-
fortnite-11560571201?mod=hp_lead_pos5.
29. Plaintiff’s innovations applicable to communication technologies
concerning multiplayer video games were the result of research and development
efforts by the inventors. The inventors recognized the need for easier, more
protected and/or secured electronic communication using multiple modes.
Plaintiff’s innovations in this regard are well-suited to allow players of multiplayer
video games to communicate with ease, privately and securely, and in real-time
using multiple modes, while keeping their contact information private. Such
innovations are protected by Plaintiff’s patents. Plaintiff’s patents were first filed
in 2006 more than a decade before Defendant’s Fortnite product was first released.
30. Beginning in November 2018, Plaintiff’s predecessor-in-interest
approached Defendant in writing to discuss a business agreement regarding
Plaintiff’s patents applicable to Defendant’s Fortnite product. Defendant did not
respond.
31. Subsequently in an October 2019 letter to Defendant, Plaintiff’s
9
predecessor-in-interest again sought a business discussion concerning Plaintiff’s
patents explaining some of the benefits of the patented technologies in online
communication. The letter included a press release, which expressly noted
“Fortnite [has] implemented [Plaintiff’s predecessor-in-interest’s] patented
technologies.” (https://www.businesswire.com/news/home/20191003005319/en/)
Defendant chose not to respond to the letter.
32. Then, in January 2020, Plaintiff’s predecessor-in-interest again
contacted Defendant in writing requesting a dialog to discuss its intellectual
property rights, even illustrating how one of Plaintiff’s patents could be infringed.
Defendant again did not respond. Despite these good-faith efforts to negotiate a
business solution, Defendant failed to show any desire for a resolution and
continues to use Plaintiff’s intellectual property without authority. In June 2021,
Plaintiff, by way of counsel, communicated in writing with Defendant alerting it of
its infringement of Plaintiff’s patents and offering to seek resolution.
Plaintiff’s U.S. Patent No. 10,708,727

33. On July 7, 2020, the United States Patent and Trademark Office duly
and legally issued United States Patent No. 10,708,727 (“the ʼ727 patent”), entitled
“Method and Apparatus to Manage Messaging Providing Different
Communication Modes Using One Identifier and Not Requiring to Disclose
10
Contact Information.” A true and correct copy of the ʼ727 patent is attached hereto
as Exhibit A.
34. Plaintiff is the owner, by assignment, of all right, title, and interest in
and to the ’727 patent, including the right to bring suit for past, present, and future
patent infringement, and to collect past, present, and future damages.
35. The claims of the ’727 patent are focused on advances over the prior
art such that their character as a whole is not directed to excluded subject matter,
such as an abstract idea, or any other subject matter excluded under 35 U.S.C.
§101.
36. The Patent Office determined all the claims of the ’727 patent to be
patent eligible, definite, novel and nonobvious.
37. The inventions claimed in the ’727 patent solve real-world,
technological problems.
38. The solutions disclosed and claimed in the ’727 patent are utilized in
computing apparatuses, including those detailed in the claim charts attached as
Exhibit E.
39. The claims of the ’727 patent recite features that address the technical
problems and challenges in the art, thereby providing specific technological
solutions that improved the state of the art. Thus, the claims are not directed to an
11
abstract concept.
40. The inventions claimed in the ’727 patent were not well-understood,
routine, or conventional as of the priority date of the ’727 patent, but instead claim
specific, novel, and nonobvious improvements to the prior art.
41. The ’727 patent is compliant with 35 U.S.C. § 101.
45. The ’727 patent is presumed valid and enforceable in accordance with
35 U.S.C. § 282.
46. Plaintiff provided Defendant with notice of its infringement of the
’727 patent prior to filing suit.

47. On November 26, 2019, the United States Patent and Trademark
Office duly and legally issued United States Patent No. 10,492,038 (“the ʼ038
patent”), entitled “Method and Apparatus to Manage Messaging Providing
Different Communication Modes Depending on One Identifier and Not Requiring
to Disclose Contact Information.” A true and correct copy of the ʼ038 patent is
attached hereto as Exhibit B.
12
49. The claims of the ʼ038 patent are focused on an advance over the prior
§101.
50. The Patent Office determined all the claims of the ʼ038 patent to be
51. The ʼ038 patent solves real-world, technological problems.
52. The solutions disclosed and claimed in the ʼ038 patent are utilized in
popular video games, including as detailed in the claim chart attached as Exhibit F.
53. The claims of the ʼ038 patent recite features that address the technical
solutions that improved the state of the art. Thus, the claims are not directed to an
abstract concept.
54. The inventions claimed in the ʼ038 patent were not well-understood,
routine, or conventional as of the priority date of the ʼ038 patent, but instead claim
13
55. The ʼ038 patent is compliant with 35 U.S.C. § 101.
59. The ʼ038 patent is presumed valid and enforceable in accordance with
35 U.S.C. § 282.
60. Plaintiff’s predecessor-in-interest provided notice to Defendant of the
allowed application that matured into the ’038 patent in a letter dated October 11,
2019 (along with related patents) and offered to reach a business agreement with
Defendant regarding its intellectual property rights. Defendant did not respond.
Plaintiff’s predecessor-in-interest provided notice to Defendant of how the ’038
patent could be infringed in a letter dated January 7, 2020, and again offered to
reach a business resolution. Defendant again did not respond. Plaintiff also
provided notice to Defendant regarding its infringement of the ’038 patent prior to
filing suit.
61. On November 27, 2018, the United States Patent and Trademark
Office duly and legally issued United States Patent No. 10,142,810 (“the ʼ810
patent”), entitled “Method and Apparatus to Manage Different Options of
14
Communications Using One User Identifier Based on Internet Protocol.” A true
and correct copy of the ʼ810 patent is attached hereto as Exhibit C.
63. The claims of the ʼ810 patent are focused on advances over the prior
§101.
64. The Patent Office determined all the claims of the ʼ810 patent to be
computing apparatuses, including as detailed in the claim chart attached as Exhibit
G.
solutions that improved the state of the art.
15
35 U.S.C. § 282.
’810 patent in a letter dated November 29, 2018, and offered to reach a business
agreement with Defendant. Plaintiff’s predecessor-in-interest again provided
Defendant with notice of the ’810 patent by letter dated October 11, 2019 (along
with related patents) and offered to reach a business agreement with Defendant
regarding its intellectual property rights. Defendant did not respond to either letter.
Plaintiff also provided Defendant with notice of its infringement of the ’810 patent
prior to filing suit.
75. On June 3, 2014, the United States Patent and Trademark Office duly
and legally issued United States Patent No. 8,744,407 (“the ʼ407 patent”), entitled
16
“Systems and Processes to Manage Multiple Modes of Communication.” A true
and correct copy of the ʼ407 patent is attached hereto as Exhibit D.
and to the ʼ407 patent, including the right to bring suit for past, present, and future
77. The claims of the ʼ407 patent are focused on advances over the prior
§101.
78. In fact, the Patent Office determined all the claims of the ʼ407 patent
to be patent eligible, definite, novel and nonobvious.
computing apparatuses, including as detailed in the claim charts attached as
Exhibit H.
solutions that improved the state of the art.
17
35 U.S.C. § 282.
’407 patent in a letter dated October 11, 2019 (along with related patents) and
offered to reach a business agreement with Defendant regarding its intellectual
property rights. Defendant did not respond. Plaintiff also provided Defendant with
notice of its infringement of the ’407 patent prior to filing suit.
COUNT ONE: DIRECT INFRINGEMENT OF THE ʼ727 PATENT
89. Plaintiff realleges and incorporates herein the preceding allegations of
this Complaint as if fully set forth herein.
90. Defendant has in the past and continue to infringe one or more claims
of the ʼ727 patent, including at least claims 1, 2, 3, 6, 7, 8, 9, 15, 16, and 17, in
violation of 35 U.S.C. §§ 271(a) by making, using, offering to sell, or selling the
18
patented invention within the United States or importing the patented invention
into the United States.
91. A representative example of Defendant’s infringing apparatuses,
methods, and systems includes (but is not limited to) Defendant’s video games
known as Fortnite. A representative claim chart demonstrating Defendant’s
infringement of the ’727 patent, either literally or under the doctrine of equivalents,
is attached as Exhibit E. Defendant’s infringing products include, without
limitation, other video games providing functionality such as that shown in the
representative charts (“Accused Products”).
92. Plaintiff notified Defendant of its infringement of the ’727 patent prior
to filing suit.
93. Plaintiff has and continues to suffer damages as a direct and
proximate result of Defendant’s direct infringement of the ’727 patent and will
suffer additional and irreparable damages unless Defendant is permanently
enjoined by this Court from continuing its infringement. Plaintiff has no adequate
remedy at law.
94. Plaintiff is entitled to: (i) damages adequate to compensate it for
Defendant’s direct infringement of the ʼ727 patent, which amounts to, at a
minimum, a reasonable royalty; (ii) attorneys’ fees; (iii) costs; and (iv) an
19
injunction.
COUNT TWO: DIRECT INFRINGEMENT OF THE ʼ038 PATENT
96. Defendant has in the past and continues to infringe one or more claims
of the ʼ038 patent, including at least claims 46, 58, 59, 60, 61, and 62, in violation
of 35 U.S.C. §§ 271(a) by making, using, offering to sell, or selling the patented
invention within the United States or importing the patented invention into the
United States.
methods, and systems include (but are not limited to) Defendant’s video games
infringement of the ʼ038 patent, either literally or under the doctrine of equivalents,
is attached as Exhibit F. Defendant’s Accused Products include, without limitation,
other video games providing functionality such as that shown in the representative
charts.
to filing suit.
20
proximate result of Defendant’s direct infringement of the ʼ038 patent and will
remedy at law.
Defendant’s direct infringement of the ʼ038 patent, which amounts to, at a
minimum, a reasonable royalty; (ii) attorneys’ fees; (iii) costs; (iv) treble damages
due to willful infringement; and (v) an injunction.
COUNT THREE: INDIRECT INFRINGEMENT OF THE ʼ038 PATENT

102. Defendant has in the past and continues to indirectly infringe at least
claims 46, 58, 59, 60, 61, and 62 in violation of 35 U.S.C. § 271(b) by actively,
knowingly, and intentionally inducing direct infringement by other persons,
including retailers, customers and/or end users, by distributing, offering for sale
and/or selling Defendant’s Accused Products in the United States and instructing
on their infringing use without authority or license from Plaintiff and in a manner
understood and intended to infringe the ʼ038 patent.
21
to filing suit.
proximate result of Defendant inducing infringement of the ʼ038 patent and will
remedy at law.
Defendant inducing infringement of the ʼ038 patent, which amounts to, at a
minimum, a reasonable royalty; (ii) attorneys’ fees; (iii) costs; (iv) treble damages
due to willful infringement; and (v) an injunction.
COUNT FOUR: DIRECT INFRINGEMENT OF THE ’810 PATENT

of the ’810 patent, including at least claims 1 and 11, in violation of 35 U.S.C. §§
271(a) by making, using, offering to sell, or selling the patented invention within
the United States or importing the patented invention into the United States.
22
is attached as Exhibit G. Defendant’s Accused Products include, without
representative charts.
to filing suit.
remedy at law.
Defendant’s direct infringement of the ’810 patent, which amounts to, at a
injunction.
COUNT FIVE: INDIRECT INFRINGEMENT OF THE ʼ810 PATENT
23
claim 11 in violation of 35 U.S.C. § 271(b) by actively, knowingly, and
intentionally inducing direct infringement by other persons, including retailers,
customers and/or end users, by distributing, offering for sale and/or selling
Defendant’s Accused Products in the United States and instructing on their
infringing use without authority or license from Plaintiff and in a manner
to filing suit.
proximate result of Defendant inducing infringement of the ’810 patent and will
remedy at law.
Defendant inducing infringement of the ’810 patent, which amounts to, at a
injunction.
24
COUNT SIX: DIRECT INFRINGEMENT OF THE ’407 PATENT
of the ’407 patent, including at least claim 53, in violation of 35 U.S.C. §§ 271(a)
by making, using, offering to sell, or selling the patented invention within the
United States or importing the patented invention into the United States.
is attached as Exhibit H. Defendant’s Accused Products include, without
representative charts.
to filing suit.
25
remedy at law.
Defendant’s direct infringement of the ’407 patent, which amounts to, at a
injunction.
COUNT SEVEN: INDIRECT INFRINGEMENT OF THE ʼ407 PATENT
claim 53 in violation of 35 U.S.C. § 271(b) by actively, knowingly, and
intentionally inducing direct infringement by other persons, including retailers,
customers and/or end users, by distributing, offering for sale and/or selling
Defendant’s Accused Products in the United States and instructing on their
infringing use without authority or license from Plaintiff and in a manner
to filing suit.
26
proximate result of Defendant inducing infringement of the ’407 patent and will
remedy at law.
Defendant inducing infringement of the ’407 patent, which amounts to, at a
injunction.
PRAYER FOR RELIEF
WHEREFORE, Plaintiff seeks the following relief:
a. Declaring that Defendant has infringed the ’727 patent, the ʼ038
patent, the ʼ810 patent, and the ’407 patent;
b. That Defendant be enjoined from further infringement of the ’727
patent, the ʼ038 patent, the ʼ810 patent, and the ’407 patent pursuant to 35 U.S.C.
§ 283;
c. That Defendant be ordered to pay damages adequate to compensate
Plaintiff for its infringement of the ’727 patent, the ʼ038 patent, the ʼ810 patent,
and the ’407 patent pursuant to 35 U.S.C. § 284;
d. That Defendant be ordered to pay prejudgment interest pursuant to 35
27
U.S.C. § 284;
e. That Defendant be ordered to pay all costs associated with this action
pursuant to 35 U.S.C. § 284;
f. That Defendant be ordered to pay Plaintiff’s attorneys’ fees pursuant
to 35 U.S.C. § 285; and
g. That Plaintiff be granted such other and additional relief as the Court
deems just and proper.
DEMAND FOR JURY TRIAL
Pursuant to Fed. R. Civ. P. 38(b), Plaintiff demands a trial by jury of all
issues so triable.
This 25th day of June, 2021.
Respectfully submitted,
/s/ Cortney S. Alexander

Stephen R. Risley (pro hac vice to be
submitted)
Telephone: 404-855-2101
Email: [email protected]
Cortney S. Alexander
Telephone: 404-855-3867
Email:
[email protected]
KENT & RISLEY LLC
5755 North Point Parkway, Ste 57
Alpharetta, Georgia 30022
Attorneys for Plaintiff
28
29
Case 6:21-cv-00663-ADA Document 1-1 Filed 06/25/21 Page 1 of 25
EXHIBIT A
US010708727B2
(12) United States Patent

Cheung et al.
( 10 ) Patent No.: US 10,708,727 B2
(45 ) Date of Patent: * Jul. 7 , 2020
(54 ) METHOD AND APPARATUS TO MANAGE 3/4365 ( 2013.01) ; H04W 4/12 (2013.01 );
MESSAGING PROVIDING DIFFERENT H04W 4/16 (2013.01) ; H04M 2201/60
COMMUNICATION MODES USING ONE ( 2013.01 );
IDENTIFIER AND NOT REQUIRING TO (Continued )
DISCLOSE CONTACT INFORMATION (58 ) Field of Classification Search
(71) Applicant: Ip Venture, Inc., San Jose, CA (US) CPC H04M 1/2473; H04L 51/32
USPC 455 /435.3 , 446 , 512 , 432.1 , 411, 518 ,
455/520
(72 ) Inventors: Kwok Wai Cheung, Tai Po (HK ); See application file for complete search history .
Peter P. Tong, Mountain View , CA
(US ); C. Douglass Thomas , Saratoga ,
CA (US) (56 ) References Cited
(73 ) Assignee : Ip Venture, Inc. , San Jose , CA (US) U.S. PATENT DOCUMENTS
5,425,516 A 6/1995 Daines
( * ) Notice : Subject to any disclaimer, the term of this 5,548,636 A 8/1996 Bannister et al .
patent is extended or adjusted under 35 (Continued )
U.S.C. 154 (b ) by 0 days.
This patent is subject to a terminal dis FOREIGN PATENT DOCUMENTS
claimer . CN 1453981 11/2003
WO WO 01/45343 A2 6/2001
(21) Appl. No.: 16 /556,205
(22 ) Filed : Aug. 29 , 2019 OTHER PUBLICATIONS
(65 ) Prior Publication Data Office Action for U.S. Appl. No. 12 /798,995, dated Apr. 11, 2011 .
US 2019/0387371 A1 Dec. 19, 2019 ( Continued )
Related U.S. Application Data Primary Examiner — Kiet M Doan
(63 ) Continuation of application No. 15 /704,181, filed on (57 ) ABSTRACT
Sep. 14 , 2017 , now Pat. No. 10,492,038 , which is a
(Continued ) A computer - implemented system and method to manage the
communication of a user are disclosed . In one embodiment,
(51) Int. Cl. when a person tries to electronically convey amessage to the
H04W 4/14 (2009.01) user, the status of the user, the identity of the person , and the
GO6Q 10/10 (2012.01 ) urgency of the message can be identified . The access priority
HOLM 3/436 (2006.01 ) of the person can be determined based on the person's
H04W 4/16 ( 2009.01 ) identity. Then , the message can be managed using one or
H04W 4/12 (2009.01) more rules and in view of the status of the user, the access
(52) U.S. CI. priority of the person and the urgency of the message.
??? H04W 4/14 (2013.01); G06Q 10/10
(2013.01 ); H04M 3/436 (2013.01 ); H04M 17 Claims, 9 Drawing Sheets
ICM Default
--- Mobile phone Voice mail
2 Office phone Voice mail
Home phone Voice mail
4 Mobile SMS/pager from mobile phone or PDA Email
5 Home office SMS (to office home PC ) Email
6 Mobile Online chat (to mobile phone or PDA ) Voice mail
7 Home Online chat (NetMeeting , AOL , ICQ etc.) Voicemail
Voice mail with instant notification to mobile devices of the user
Voice mail without notification to mobile devices
Office fax
Home fax Reject
Mobile Email ( Blackberry etc.) Email
13 Email Reject
User defined
US 10,708,727 B2
Page 2
Related U.S. Application Data 9,555,334

9,736,664
B2 1/2017 Bernard et al.
B2 8/2017 Cheung et al.
continuation of application No. 15 /469,440 , filed on 10,142,810 B2 11/2018 Cheung et al.
Mar. 24 , 2017 , now Pat . No. 10,142,810 , which is a 10,183,219 B2 1/2019 Linden et al .
continuation of application No. 14 /922,344, filed on 10,207,191 B2 2/2019 Jensen
Oct. 26 , 2015 , now Pat. No. 9,736,664 , which is a 10,492,038 B2 11/2019 Cheung et al.
2001/0011014 Al 8/2001 Higuchi et al.
continuation of application No. 14 /272,632 , filed on 2001/0014611 A1 8/2001 Dufort
May 8 , 2014 , now Pat. No. 9,204,268, which is a 2001/0028709 Al 10/2001 Makela et al.
continuation of application No. 12 /798,995 , filed on 2001/0031633 A1 10/2001 Tuomela et al.
Apr. 14 , 2010 , now Pat. No. 8,744,407 , which is a 2002/0067806 Al 6/2002 Rodriguez et al.
continuation of application No. 11/452,115 , filed on 2002/0073207 Al 6/2002 Widger et al.
2002/0094067 Al 7/2002 August
Jun . 12 , 2006 , now Pat. No. 7,729,688 , which is a 2002/0142756 Al 10/2002 Rutledge et al.
continuation -in -part of application No. 11 /006,343, 2002/0181672
2003/0039339
A1 12/2002 Cannell et al .
A1 2/2003 Luehrig et al.
filed on Dec. 7, 2004 , now Pat. No. 7,116,976 . 2003/0041048 A1 2/2003 Balasuriya
2003/0065779 Al 4/2003 Malik et al.
(60) Provisional application No.60/ 527,565, filed on Dec. 2003/0103600 A1 6/2003 Potter
8 , 2003, provisional application No.60/689,686 , filed 2003/0105854 Al 6/2003 Thorsteinsson et al .
on Jun . 10 , 2005 . 2003/0112948 Al 6/2003 Brown et al .
2003/0129968 Al 7/2003 Earl
2003/0191676 Al 10/2003 Templeton
(52) U.S. Ci. 2003/0191814 Al 10/2003 Tran
CPC H04M 2203/2011 (2013.01 ); H04M 2003/0232629 A1 12/2003 Jang et al.
2203/651 ( 2013.01); H04M 2207/18 (2013.01) 2004/0024882 A1 2/2004 Austin et al.
2004/0072585 Al 4/2004 Le et al .
(56 ) References Cited 2004/0078340 A1 4/2004 Evans
2004/0122979 Al 6/2004 Kirkland
U.S. PATENT DOCUMENTS 2004/0143667 Al 7/2004 Jerome
2004/0203794 Al 10/2004 Brown et al .
2004/0203919 A1 10/2004 Ross et al.
5,610,970 A 3/1997 Fuller et al. 2004/0240650 Al 12/2004 Bear et al.
5,752,191 A 5/1998 Fuller et al . 2004/0248596 Al 12/2004 Panchal
5,758,079 A 5/1998 Ludwig et al. 2005/0020288 A1 1/2005 Davis
5,828,731 A 10/1998 Szlam et al. 2005/0027385 Al 2/2005 Yuch
5,930,700 A 7/1999 Pepper et al. 2005/0037785 Al 2/2005 Chen
5,970,388 A 10/1999 Will 2005/0038690 A1 2/2005 Hayes-Roth
6,119,022 A 9/2000 Osborn et al. 2005/0071253 Al 3/2005 Yang
6,327,628 B1 12/2001 Anuff et al . 2005/0107130 A1 5/2005 Peterson , II
6,359,892 B1 3/2002 Szlam 2005/0136955 Al 6/2005 Mumick et al .
6,463,462 B1 10/2002 Smith et al . 2005/0191994 Al 9/2005 May et al.
6,577,859 B1 6/2003 Zahavi et al. 2005/0192061 A1 9/2005 May et al.
6,636,888 B1 10/2003 Bookspan et al. 2005/0273327 A1 12/2005 Krishnan
6,665,534 B1 12/2003 Conklin et al. 2006/0003803 A1 1/2006 Thomas et al.
6,788,766 B2 9/2004 Logan 2006/0075038 Al 4/2006 Mason et al.
6,801,793 B1 10/2004 Aarnio et al . 2006/0168054 Al 7/2006 Burkhart et al.
6,816,578 B1 11/2004 Kredo et al . 2006/0212561 A1 9/2006 Feng
6,819,757 B1 11/2004 Cook et al. 2006/0239419 Al 10/2006 Joseph et al.
6,819,945 B1 11/2004 Chow et al . 2006/0259565 Al 11/2006 Cheung et al .
6,978,136 B2 12/2005 Jenniges et al. 2006/0276210 Al 12/2006 Thomas et al.
7,010,288 B2 3/2006 Brown et al. 2006/0288099 A1 12/2006 Jefferson et al.
7,010,332 B1 3/2006 Irvin et al . 2007/0005368 A1 1/2007 Chutorash et al.
7,027,842 B2 4/2006 Zhang et al. 2007/0047522 A1 3/2007 Jefferson et al.
7,043,261 B2 5/2006 Krishnan 2007/0238474 Al 10/2007 Ballas
7,072,452 B1 7/2006 Roberts et al . 2008/0261636 A1 10/2008 Lau et al.
7,085,253 B2 8/2006 Yang 2010/0114958 Al 5/2010 Korenshtein
7,107,010 B2 9/2006 Heinonen et al. 2010/0205272 Al 8/2010 Cheung et al.
7,110,963 B2 9/2006 Negreiro 2011/0151582 A1 6/2011 Basile
7,111,044 B2 9/2006 Lee 2011/0151852 A1 6/2011 Olincy et al .
7,116,976 B2 10/2006 Thomas et al. 2014/0242956 A1 8/2014 Cheung et al.
7,188,073 B1 3/2007 Tam et al . 2014/0256293 Al 9/2014 Thomas et al.
7,224,775 B1 5/2007 Shaffer et al. 2016/0044474 Al 2/2016 Cheung et al.
7,317,706 B1 1/2008 Hao et al. 2017/0201872 A1 7/2017 Cheung et al.
7,346,630 B2 3/2008 Eichstaedt et al. 2018/0014169 Al 1/2018 Cheung et al.
7,376,434 B2 5/2008 Thomas et al.
7,403,972 B1 7/2008 Lau et al.
7,686,693 B2 3/2010 Danieli et al. OTHER PUBLICATIONS
7,729,688 B2 6/2010 Cheung et al.
7,792,552 B2 9/2010 Thomas et al . Notice of Allowance for U.S. Appl. No. 12 /798,995 dated Sep. 29 ,
7,890,128 B1 2/2011 Thomas et al. 2011.
8,112,104 B1 2/2012 Thomas et al. Notice of Allowance for U.S. Appl. No. 12 /798,995 dated Jan. 3 ,
8,280,419 B1 10/2012 Thomas et al. 2012 .
8,353,773 B2 1/2013 Sasaki et al.
Notice of Allowance for U.S. Appl. No. 12,798,995, dated May 9 ,
8,391,459 B2 3/2013 Jackson et al . 2012 .
8,429,231 B2 4/2013 Wu et al.
8,737,978 B1 5/2014 Thomas et al. Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Oct. 3 ,
8,744,407 B2 6/2014 Cheung et al. 2012 .
8,827,811 B2 9/2014 Kim et al . Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Jul. 16 ,
9,204,268 B2 12/2015 Cheung et al. 2013 .
US 10,708,727 B2
Page 3
( 56 ) References Cited " Our Solution," lotum Corp., http://www.iotum.com/our_solution .

php , downloaded May 15 , 2006 , pp . 1-2.
OTHER PUBLICATIONS Short Message Service/Interactive Voice Response ( SMS/IVR ),
Lucent Technologies, 2003 , pp . 1-2 .
Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Oct. 30 , “ Text messaging” , Vodafone Group, 2001, Vodafone Services ,
2013 . “ All about textmessaging” , http://www.vodafone.co.nz/services/07 .
Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Feb. 20 , a.l_two_way_messaging.jsp ?hd =4yourbusiness& downloaded
2014 . Oct. 14 , 2003 , pp . 1-2 .
Office Action for U.S. Appl. No. 14 /272,632, dated Jul. 27 , 2015. “ We bring relevance to communications,” CNET News, Ina Fried ,
Notice of Allowance for U.S. Appl. No. 14 /272,632 , dated Sep. 18 , Jul. 21, 2005, pp . 1- 2 .
2015 .
Office Action for U.S. Appl. No. 14 /922,344 , dated Apr. 27, 2016 . Appenzeller, et al., “ The Mobile People Architecture” , Technical
Office Action for U.S. Appl. No. 14 /922,344 , dated Oct. 7, 2016 . Report : CSL - TR -00000 , Computer Systems Laboratory, Depart
Notice ofAllowance for U.S. Appl. No. 14 / 922,344 , dated Feb. 14 , ments of Electrical Engineering and Computer Science , Stanford
2017 . University, Jan. 1999 , pp . 1-13 .
Notice of Allowance for U.S. Appl. No. 14 /922,344, dated Mar. 6 , BlackBerry, “ Voice and SMS” , http://www.blackberry.com/products/
2017 . service /voices_sms.shtml?DCPID = hmsvoice downloaded Oct. 2,
Notice of Allowance for U.S. Appl. No. 15 /469,440 , dated Sep. 6 , 2003, p . 1 .
2017 . Bulk , F. “ Final Project: Skype,” http://www1.cs.columbia.edu/
Notice of Allowance for U.S. Appl. No. 15 /469,440 , dated Dec. 19 , ~ salman / skype/frank.pdf, May 5 , 2004 , pp . 23.
2017 . Calsyn ,Martin and Desseault , Lisa, “ Presence Information Protocol
Notice of Allowance for U.S. Appl. No. 15/469,440 , dated Aug. 8 , Requirements,” Internet Draft, Feb. 9 , 1998, pp . 1-27 .
2018 . Emergin Inc., “Emergin WirelessOffice 5.0” , http ://www.emergin .
Notice of Allowance for U.S. Appl.No. 15 / 469,440, dated Sep. 27, com / ? source = overture , downloaded Oct. 2 , 2003, p . 1.
2018 . Fastmobile Inc., “ Dialog GSM launches Push ‘ n ’ Talk walkie talkie
Notice of Allowance for U.S. Appl. No. 15 / 704,181, dated Apr. 27, service Push to Talk over Cellular Now in Sri Lanka Dialog GSM
2018 . Pioneers Latest GSM Advancement” , Press Release , Dec. 1, 2004,
Notice of Allowance for U.S. Appl. No. 15/704,181, dated Aug. 22 , pp. 1-2.
2018 . Fastmobile, “ fastmobile's fastchatTM Instant Communications Appli
Notice of Allowance for U.S. Appl. No. 15 /704,181, dated Nov. 6 , cation is Coming to Thousands of Mobile Phone Retail Stores
2018 . Nationwide” , fastmobile Press Release , Sep. 15 , 2003, pp . 1-3 .
Notice of Allowance for U.S. Appl. No. 15/ 704,181, dated Jul. 2 , IMBOT, Press Release , “ IMBOT offers new Text 2 Voice Service
2019 . Text 2 Voice service enables wireless customers to send voice
First Office Action for CN Patent Application No. 200680027964.9 , messages from 2 -Way devices ” , Oct. 29 , 2001, pp. 1-2 .
dated Mar. 26 , 2010 ( 17 pages ). Internet Traveler, “ Welcome to the Inter.Net Communicator Tour !" ,
Second Office Action for CN Patent Application No. 200680027964 . http://www.inter.net/traveler/tour/communicator_messaging.php, down
9 , dated Oct. 25 , 2010 ( 14 pages). loaded Oct. 14 , 2003, p . 1 .
Third Office Action for CN Patent Application No. 200680027964. J. Rosenberg, H. Schulzrinne, Internet Draft, “ SIP for Presence,"
9 , dated Apr. 8, 2011 ( 11 pages). http://www.alternic.org/drafts/drafts-r-s/draft-rosenberg=sip-pip-00 .
Notice of Rejection for CN Patent Application No. 200680027964 . txt, Nov. 13, 1998 , Bell Laboratories, Columbia , pp . 1-31 .
9 , dated Jan. 6 , 2012 ( 11 pgs .). Joseph , Anthony D. et al., “ The Case for Services over Cascaded
AIM Buddy List, https://web.archive.org/web/20050531083304/ Networks” , EECS Department, CS Division , University of Califor
http://www.aim.com/help_faq/starting_out/buddylist.adp?aolp= , down nia , Berkeley, http://iceberg.cs.berkeley.edu/, International Confer
loaded Aug. 9 , 2019 , 1 pg. ence on Wireless and Mobile Multimedia 1998 , pp . 1-9 .
AIM® Inside the Sidekick , https://web.archive.org/web/ MSN Messenger Version 7.0 , “ Messenger,” https://web.archive.org/
20050519114310/http :/mymobile.aol.com/portal/im/pdfs/tmobile/ web / 20050601023444 /http :/messenger.msn.com downloaded
AIM_SIDEKICK_UG_tmobile.pdf, copyright 2002 by America Aug. 6 , 2019 , 2 pgs .
Online, Inc., 13 pgs. MobileShop , “ SMS also know as text messaging” , http : //www .
AIM Registration , https://web.archive.org/web/20041216085601/ mobileshop.org/howitworks.sms.htm , downloaded Oct. 14 , 2003,
http :/www.aim.com :80 /help_faq/starting_out/registration.adp ? pp. 1-2.
aolp = , downloaded Aug. 9 , 2019 , 2 pgs. MSN Messenger:mac , https://web.archive.org/web/20050604080622/
" AOL ® Instant Messenger TM ,” https://web.archive.org/web/ http://www.microsoft.com/mac/default.aspx?pid=msnmessenger,down
20050601001345 /htts:/www.aim.com/, downloaded Aug. 6 , 2019 , 2 loaded Aug. 9 , 2019 , 2 pgs .
pgs. MSN Messenger, “ Communicate with MSN® Messenger," https: //
" Company Overview ”, http://www.fastmobile.com/company_overview . web.archive.org/web/20051013055708/http://www.imagine-msn .
html, downloaded Nov. 5, 2003, p . 1 . com /messenger/post/communicate/instantmessage.aspx, down
Download AIM for Windows, https://web.archive.org/web/ loaded Aug. 9 , 2019 , 2 pgs.
20050204020358 /http :/channels.netscape.com/wrap/linker.jsp ? floc = MSN Messenger," Download MSN Messenger," https: //web.archive.
at_oswin_1_12 & ref = http ://www.aim.com/get_aim/win/latest_win . org/web /20050601002632 /http : //messenger.msn.com/download/, down
adp ?aolp = # whatsnew , downloaded Aug. 9 , 2019 , 2 pgs. loaded Aug. 9 , 2019 , 1 pg .
" Introducing the Tellme Voice Application Network” , Tellme, http :// MSN Messenger, “ Most Frequently Asked Questions," https: //web .
www.tellme.com/products/ , downloaded Oct. 2 , 2003, p . 1 . archive.org/web/20050601014205/http://messenger.msn.com/
“ iotum History,” Iotum Corp., http://iotum.com/simplyrelevant/2006/ Help /, downloaded Aug. 9, 2019 , 8 pgs .
04 /03/iotum -historyl, downloaded May 15 , 2006 , pp. 1-4 . Schulzrinne , H. et al., RPID : Rich Presence Extensions to the
“Messaging ", Vodafone Group , 2001, http :www.vodafone.co.nz/ Presence Information Data Format (PIDF), draft-ietf= simple-rpid
business/ 10.2.3_messaging.jsp , downloaded Oct. 14 , 2003 , pp . 1-2. 06.txt, Jun. 2 , 2005, http://www1.ietf.org/mail-archive/web/simple/
“ Microsoft Windows Messenger: Go Beyond Text with Voice & current/msg05398.html, downloaded Nov. 15 , 2006 , pp . 1-35 .
Video Chats” , Dell Inc., http://www.dell.com/us/en/dhs/topics/ Schulzrinne , H. et al., RPID : Rich Presence Extensions to the
segtopic_002_xp_im.htm , downloaded Oct. 2 , 2003 , pp . 1-2 . Presence Information Data Format (PIDF), draft -ietf-simple- rpid
“ Microsoft WindowsMessenger: Instantly Communicate with Fam 10.txt, Dec. 20 , 2005 , pp . 1-41.
ily and Friends Messenger” , Dell Inc., http://www.dell.com/us/en/ Schulzrinne , H. et al., RPID : Rich Presence Extensions to the
dhs/topics/segtopic_001_xp_im.htm , downloaded Oct. 2, 2003, pp . Presence Information Data Format (PIDF), draft -ietf-simple -rpid
1-3 . 10.txt, Dec. 4 , 2005, pp . 1-35 .
US 10,708,727 B2
Page 4
( 56 ) References Cited Symbian Ltd., “ Technology : Why is a different operating system

needed ”, http://www.symbian.com/technology/why-diff-os.html,down
OTHER PUBLICATIONS loaded Nov. 5 , 2003 , pp . 1-5 .
SkypeTM , “ Skype is free Internet telephony that just works,” https:// Using AIM on Windows— Video IM , https://web.archive.org/web/
web.archive.org/web/20050601003206/http:/www.skype.com , down 20050307231707 /http :/www.aim.com/help_faq/using/win/video_im .
loaded Aug. 6 , 2019 , 3 pgs. adp ? aolp = , downloaded Aug. 9, 2019 , 2 pgs.
SkypeTM , “ Skype is free Internet telephony that just works,” https: // Using AIM on Windows — Mar. 8 , 2005 , https://web.archive.org
web.archive.org/web/20050601003206/http://www.skype.com/, down web / 20050308012857 /http : //www.aim.com/help_faq/using/win/
loaded Aug. 9 , 2019 , 21 pgs . aimtalk.adp ?aolp =, downloaded Aug. 9, 2019 , 2 pgs.
SkypeTM , “ How to Use Skype," https://web.archive.org/web/ Using AIM on Windows- Mar. 5 , 2005, https://web.archive.org/
20041229163311/http:/www.skype.com/help/guides/usingskype . web /20050305095328/http :/www.aim.com :80 /help_faq /using/win/
html, downloaded Aug. 9 , 2019 , 6 pgs . instant_message.adp ? aolp = , downloaded Aug. 9 , 2019 , 2 pgs .
SkypeTM , “ How to Add a Contact User Guide,” https: //web.archive . Verizon Wireless,“ TXT messaging” , http://www.vtext.com/customer_
org/web /20041230160101/http :/www.skype.com/help/guides/adduser. site /jsp/messaging_lo.jsp , downloaded Oct. 2 , 2003, p . 1.
html, 10 pgs. W3C , Voice Extensible Markup Language (VoiceXML) Version
SkypeTM , “ Registering a Skype Name User Guide,” https://web . 2.0 , W3C , www.w3.org, Feb. 20 , 2003.
archive.org/web/20041229042631/http:/www.skype.com/help/guides/
registration.html, 10 pgs . Yahoo!Messenger, “ Yahoo!Messenger Talk for Free !" , http ://
Sonim Technologies, Inc., “ Integrated voice and text messanging messenger.yahoo.com/messenger/help/voicechat.html, downloaded
over GPRS showcased jointly by Sonim , Symbian and Texas Oct. 2 , 2003 , pp . 1-2 .
Instruments” , Sonim Press Release, Dec. 2 , 2002, pp . 1-2 . “ Yahoo!Messenger,” https://web.archive.org/web/20050601012258/
Symbian Ltd., “ Symbian OS Version 7.0 : Functional description ” , http :/messenger.yahoo.com , downloaded Aug. 6 , 2019, 1 pg .
Revision 1.5 , Feb. 2003, pp . 1-24 . “ Yahoo!Messenger," https://web.archive.org/web/20050601042101/
Symbian Ltd., “ Symbian OS Version 7.0s: Functional description ” , http://messenger.yahoo.com/newtoim.php ,downloaded Aug. 9, 2019 ,
Revision 2.1, Jun . 2003, pp . 1-29. 3 pgs.
Symbian Ltd., “ Technology : Creating Symbian OS phones ” , http : // “ Yahoo! Help - All-New Messenger 6.0 ," https://web.archive.org
www.symbian.com/technology/create-symb-OS-phones.html, down web /20040806142252/http : //help.yahoo.com/help/us/messenger/win/
loaded Nov. 5 , 2003 , p . 1-8 . abuse /abuse -02.html, downloaded Aug. 9 , 2019 , 2 pgs .
U.S. Patent Jul. 7 , 2020 Sheet 1 of 9 US 10,708,727 B2
Default
???? Mobile phone Voice mail
2 Office phone Voice mail
3 Home phone Voice mail
5 Home/office SMS ( to office /home PC ) Email
6 Mobile Online chat (to mobile phone or PDA ) Voice mail
HomeOnline chat (Net Meeting, AOL , ICQ etc.) Voice mail
8 Voice mail with instant notification to mobile devices of the user
10 Office fax
Home fax Reject
12 Mobile Email (Blackberry etc.) Email
13 Email Reject
User defined
FIGURE 1
ContactClassi Kinship family members , love ones

ContactClass 2 Relatives and friends
ContactClass 3 Boss and VIP
ContactClass4 Colleagues
ContactClass5 Subordinates
ContactClass6 Business acquaintances
ContactClass7 VIP Clients
ContactClass 8 Clients
ContactClass9 Secretary
ContactClass 10 User defined
UrgClass 1 Life threatening - interrupt at any time and occasion

UrgClass2 Urgent confirmed meeting reminder -- interruption
allowed
UrgClass3 Urgentmatter requiring immediate attention
UrgClass4 Importantmatter requiring quick attention
UrgClass ) Regular work related matter
UrgClass Casual contact
UrgClass 7 Cold calls from unknown person
UrgClass8 User defined
FIGURE 3
MyBusyStatel Importantmeeting
MyBusy State2 Ordinary meeting
MyBusyState3 Available
MyBusy State4 Sleeping
MyBusy State ) Resting
MyBusy Stateo User defined
FIGURE 4
ContactClass UrgClass MyBusyState ICM allowed

ContactClass2 UrgClass 1-3 All
UrgClass 4-6 MyBusyStatel All
MyBusyState2-3
MyBusyState4-5 All
UrgClass 7-8 ICM 13
100
106
102
NETWORK
lol
u 108
104
104
106 102
FIG . 6

200
START
202
INCOMING
VOICE
CALL NO
?
YES
204
206 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE
CALL 212
AUDIO
MESSAGE 214
YES
ENGAGE IN ?
VOICE CALL OBTAIN AND SEND
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 216
NO ? TEXT
218
MESSAGE
YES ? YES
OBTAIN AND SEND
TEXT MESSAGE
NO TO CALLER
220
VOICE 222
MAIL YES
? DIRECT TO
NO VOICE MAIL
DROP THE INCOMING

VOICE CALL 224
FIG . 7
END
300
START
ANSWER THE INCOMING VOICE CALL 302
PRESENT ONE OR MORE PREDETERMINED

AUDIO MESSAGES 304
306
PREDET.
AUDIO
MESSAGE NO
SELECTED 310
?
CUSTOM
YES
AUDIO
308 MESSAGE
REQUESTED NO
?
PLAY SELECTED AUDIO
MESSAGE FOR CALLER
YES
RECORD CUSTOM AUDIO

312
MESSAGE
PLAY CUSTOM AUDIO

MESSAGE FOR CALLER 314
CLOSE THE INCOMING VOICE CALL 316
END
FIG . 8
400
START
DROP THE INCOMING VOICE CALL 402
DISPLAY ONE OR MORE PREDETERMINED

TEXT MESSAGES 404
406
PREDET .
TEXT
MESSAGE NO
SELECTED 410
?
CUSTOM
YES
TEXT
408 MESSAGE
REQUESTED NO
TRANSMIT SELECTED ?
TEXT MESSAGE
TO CALLER
YES
ENTER CUSTOM
TEXT MESSAGE 412
TRANSMIT CUSTOM
TEXT MESSAGE 414
TO CALLER
END
FIG . 9
START
500
202
INCOMING
VOICE
CALL NO
?
YES
204
206 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE
CALL 502
HEAD - SET 214

ACTIVE YES
ENGAGE IN ?
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 504
NO ?
DISPLAY 218
YES PRESENT YES
? OBTAIN AND SEND
TEXT MESSAGE
NO TO CALLER
220
VOICE 222
MAIL YES
? DIRECT TO
NO VOICE MAIL
DROP THE INCOMING

224
FIG . 10 VOICE CALL
END
600
START
602
INCOMING
TEXT
MESSAGE NO
?
YES
604
AUDIO
OR TEXT
PRESENTATION 610
TEXT
?
DISPLAY THE INCOMING
TEXT MESSAGE
AUDIO
606 CONVERT THE INCOMING TEXT

MESSAGE TO AN AUDIO MESSAGE
608 PLAY THE AUDIO MESSAGE
END
FIG . 11
START
700
702
REPLY
MESSAGE
NO TO BE
SENT 714
?
YES DISPLAY TEXT
ENTRY SCREEN
704
AUDIO
OR TEXT
MESSAGE TEXT
TO BE MESSAGE
TEXT ENTERED NO
FORMED
? ?
706 716 YES

AUDIO
PROMPT FOR AUDIO MESSAGE
708
AUDIO
MESSAGE
RECORDED NO
?
710
YES
CONVERT THE AUDIO MESSAGE

TO A TEXT MESSAGE
TRANSMIT THE TEXT MESSAGE

OVER A WIRELESS NETWORK 712
END FIG . 12
US 10,708,727 B2
1 2
METHOD AND APPARATUS TO MANAGE than one phone number and multiple electronic mail
MESSAGING PROVIDING DIFFERENT addresses. There are people we like to communicate with ,
COMMUNICATION MODES USING ONE and there are those we prefer to avoid . Managing informa
IDENTIFIER AND NOT REQUIRING TO tion from all such different modes can be quite time con
DISCLOSE CONTACT INFORMATION 5 suming.
It should be apparent from the foregoing that there is still
CROSS -REFERENCE TO RELATED a need to help manage the numerous modes of communi
APPLICATIONS cation .
This application is a continuation of U.S. patent applica- 10 SUMMARY OF THE INVENTION
tion Ser. No. 15 /704,181 , filed Sep. 14 , 2017 , and entitled
"METHOD AND APPARATUS TO MANAGE MESSAG Different embodiments of a computer-implemented sys
ING PROVIDING DIFFERENT OPTIONS OF COMMU tem and method to manage the communication of a user are
NICATION DEPENDING ON ONE IDENTIFIER AND disclosed . A person tries to electronically convey a message
NOT REQUIRING TO DISCLOSE CONTACT INFOR- 15 to the user. In one embodiment, the status of the user is
MATION ,” which is hereby incorporated herein by refer identified ; the identity of the person is identified ; the urgency
ence , which application is a continuation of U.S. patent of the message is identified ; the access priority of the person
application Ser. No. 15 /469,440 , filed Mar. 24 , 2017, now is determined based on the person's identity ; and a process
U.S. Pat. No. 10,142,810 , and entitled “ A NETWORD is set to manage the message using one or more rules, and
BASED PORTAL TO MANAGE COMMUNICATION ,”, 20 in view of the status of the user , the access priority of the
which is hereby incorporated herein by reference, which person and the urgency of the message .
application is a continuation of U.S. patent application Ser. Based on different embodiments, the status of the user
No. 14/921 44 , filed Oct. 26 , 2015 , now U.S. Pat. No. depends on the current activity or location of the user, or the
9,736,664, and entitled “ SYSTEMS AND PROCESSES TO current time. The status of the user can also be defined by the
MANAGE MULTIPLE MODES OF COMMUNICA- 25 user. Similarly , the access priority of the person can be
TION ,” which is hereby incorporated herein by reference , defined by the user, or is set depending on the user's reaction
which application is a continuation of U.S. patent applica towards a prior message from the person . Also , the urgency
tion Ser. No. 14 /272,632 , filed May 8 , 2014 , now U.S. Pat. of the message is set by the person.
No. 9,204,268 , and entitled “ SYSTEMS AND PROCESSES The process can depend on themode of communication of
TO MANAGE MULTIPLE MODES OF COMMUNICA- 30 the message . For example, the mode of communication can
TION ,” which is hereby incorporated herein by reference, include a mobile phone, an office phone , a home phone, a
which application is a continuation of U.S. patent applica mobile SMS, a pager from a mobile phone or PDA , a
tion Ser. No. 12 /798,995 , filed Apr. 14 , 2010 , now U.S. Pat. home/office SMS, mobile online chat, home online chat, a
No.8,744,407, and entitled “SYSTEMSAND PROCESSES voice mail with /without instant notification , an office fax , a
TO MANAGE MULTIPLE MODES OF COMMUNICA- 35 home fax , a mobile email , and an email.
TION ,” which is hereby incorporated herein by reference , In one embodiment, the user receives the message
which application is a continuation of U.S. patent applica through a handheld device , such as a cellular phone . In
tion Ser. No. 11 /452,115 , filed Jun. 12 , 2006 , now U.S. Pat. another embodiment, the message is electronically conveyed
No.7,729,688 , and entitled “ SYSTEMS AND PROCESSES based on Internet protocol through a website .
TO MANAGE MULTIPLE MODES OF COMMUNICA- 40 In one embodiment, though the process allows the user to
TION ” , which is hereby incorporated herein by reference , receive the message , the person is not aware of the contact
which application is a continuation - in -part application of information of the user. For example , the person is not aware
U.S. patent application Ser. No. 11/006,343 , filed Dec. 7 , of the phone number of the cellular phone that the user used
2004, now U.S. Pat . No. 7,116,976 , and entitled “ ADAPT to talk to the person . This prevents the person from directly
ABLE COMMUNICATION TECHNIQUES FOR ELEC- 45 accessing the user without going through an intermediate
TRONIC DEVICES ,” which is hereby incorporated herein control, such as a website . Similarly , the user does not have
by reference , which claims priority to U.S. Provisional to be aware of the contact information of the person .
Patent Application No. 60 /527,565, filed Dec. 8 , 2003 , In another embodiment, the defined access priority of the
entitled " ADAPTABLE COMMUNICATION TECH person is stored at a website , allowing the website to access
NIQUES FOR ELECTRONIC DEVICES,” and which is 50 such information without asking for the user's permission .
hereby incorporated herein by reference . In one embodiment, the defined access priority is stored in
This application , by way of U.S. patent application Ser. a private database under the user's control.
No. 11 /452,115 , also claims priority to U.S. Provisional In one embodiment, textmessages could be received in an
Patent Application No. 60 /689,686 , filed Jun . 10 , 2005, audio manner, and audio messages could be sent as text
entitled " SYSTEMS AND PROCESSES TO MANAGE 55 messages.
MULTIPLE MODES OF COMMUNICATION ,” and which Other aspects and advantages of the present invention will
is hereby incorporated herein by reference . become apparent from the following detailed description ,
which , when taken in conjunction with the accompanying
BACKGROUND OF THE INVENTION drawings, illustrates by way of example the principles of the
60 invention .
For many years , other than mails from post offices, we
typically only received information from afar through tele BRIEF DESCRIPTION OF THE DRAWINGS
phones. However, in the past few years, ways that others can
send us information have increased significantly. Just to list FIG . 1 shows a number of intelligent communication
a few differentmodes of communication , we can be reached 65 modes according to one embodiment of the invention.
from standard desk phones, fax , cell phones, electronic FIG . 2 shows a number of contact classes according to
mails, and instant messages. In addition , we can have more one embodiment of the invention .
US 10,708,727 B2
3 4
FIG . 3 shows a number of urgency classes according to A number of embodiments depend on the differentmodes
one embodiment of the invention . of communication converging onto the internet protocol
FIG . 4 shows a number of statuses of a user according to platform . A communication gateway or a portal is formed
one embodiment of the invention . allowing the user to receive communications from numerous
FIG . 5 shows one embodiment of an example of an 5 sources through different modes. This, in turn , could reduce
Access Priority Database according to one embodiment of the numerous addresses the user has to remember, to one
the invention . address. For example , an e -mail address for the user can
FIG . 6 is a communication system according to one serve as an access identifier for the different communication
embodiment of the invention . addresses from different communication modes. The access
FIG . 7 is a flow diagram of a personal call response 10 identifier can become the user's digital identity . In one
process according to one embodiment of the invention. embodiment, the user's other types of identification , such as
FIG . 8 is a flow diagram of an audio message response the user's driver licenser number, can be the user's access
process according to one embodiment of the invention . identifier.
FIG . 9 is a flow diagram of a text message response One embodiment of the invention uses an open portal
process according to one embodiment of the invention . 15 based on the web . Based on the portal, the user can securely
FIG . 10 is a flow diagram of an automated call response determine who can reach him at what conditions . This can
process according to one embodiment of the invention . be done based on a status indicator. As an example , this
FIG . 11 is a flow diagram of a message presentation indicator is determined according to the status of the user,
process according to one embodiment of the invention. the access priorities of the person trying to reach the user (or
FIG . 12 is a flow diagram of a message presentation 20 the relationship or the lack of relationship between the user
process according to one embodiment of the invention . and the person ), and /or the urgency of the message from the
Same numerals in FIGS. 1-12 are assigned to similar person . The status of the user can be dynamically deter
elements in all the figures. Embodiments of the invention are mined , based on the current condition (s) of the user. The
discussed below with reference to FIGS. 1-12 . However, portal can allow the user and the person to select different
those skilled in the art will readily appreciate that the 25 options,which can be modified as desired. For example, the
detailed description given herein with respect to these fig relationship can be preset by the user and stored in a
ures is for explanatory purposes as the invention extends database , while the urgency of the message can be setby the
beyond these limited embodiments . person .
Thus , in one embodiment, the portal can be used to
DETAILED DESCRIPTION OF THE 30 control the selection and setting of different intelligent
INVENTION communication modes for the user . These intelligent com
One embodiment of the invention can automatically
munication modes allow priorities ofvarious kinds of com
munication options to be by the user. The portal allows
remove unwanted communications. Certain communica worldwide access to the user , and can dynamically deter
tions are relatively easy to determine to be unwanted , such 35 mine, for example, whether a call initiated at different time
as marketing cold calls and wrong number calls. Other by different callers should be accepted by the user in
communicationsmay be more difficult. They can depend not real-time or handled by other mechanisms. From this infor
just on the sources of the communication , but also the mation , communication requests can be classified , for
conditions or status of the receiver (a user ) of the commu example , into different degrees of undesirability . Some
nication . The status can be related to the user's current 40 requests can be automatically blocked from the user. Others
activity and /or location . For example, when the user is on a can be diverted and handled by other mechanism , such as
train going to work , the user probably does not mind diverting a phone call to an email or voice mail.
chatting with his grandchild . However , if the user is having In one embodiment, the portal or gateway also includes a
his yearly review meeting with his boss , the user probably database to keep track of the user's different contacts or
would prefer to avoid the call from his grandchild , unless it 45 acquaintances, and the access priorities of each contact . The
is an emergency . Based on the embodiment, communica user can modify information in the database , such as assign
tions from sources the user wants to postpone receiving can ing and/or changing the priorities of the contacts. Based on
be automatically diverted . the information (or lack of information ) in the database of
In one embodiment, the user can get appropriate notifi the contact trying to access the user, and based on the status
cation on the source of the incoming communication 50 of the user, the gateway can automatically select an intelli
request . The attributes of the notification can depend on the gentmode of communication for the user . This selection can
urgency of the communication and/or the status of the user. be done dynamically .
The user may receive information from differentmodes of In one embodiment, the portal can dynamically change
communication . For example , the user can have mobile the access priorities of a caller trying to reach the user. For
phones , fixed lines at home or office, emails, SMS , and 55 example, previously the caller is ofhigh priority to the user,
faxes, with their different numbers and / or addresses. One and the user has set her access priorities accordingly. Lately ,
embodiment can help the user efficiently manage informa every time the caller trying to reach the user, the request was
tion from the differentmodes . The user only has to remem denied . After a preset number of rejections, the portal can
ber one specific address from one mode of communication . automatically send a message to the user, asking the user if
Through that address , the user can receive communications 60 the user would like to lower the access priority of the caller.
from all modes of communication , independentofwhere the If the response is affirmative, the caller's priority is auto
user is , or the type of hardware the user has . This allows the matically reduced .
user to efficiently maintain his communication from the In another embodiment, the user does not have to set
numerous modes even when he is traveling . For example , priorities of each contact. The system monitors every call ,
the user does not have to change phones (and the phone 65 and provides the contact’s identity to the user. Based on the
numbers ) when he moves from areas covering 3G to areas user's reaction to the call (e.g. accepting or rejecting it ), the
that do not. system automatically sets the contact's priorities . In one
US 10,708,727 B2
5 6
embodiment, the system can then query the user for approval based on tables in FIGS. 1-4 , the user can set up an Access
on the setting , and allow the user to adjust it as necessary. In Priority Database for different ContactClasses, as shown in
another embodiment, the system can continue to modify the FIG . 5 .
caller's priorities based on the user's reaction to the caller's As another example , the user can categorize the following
subsequent calls. 5 contacts into the corresponding ContactClasses:
In one embodiment, the user could keep information he
believes to be sensitive local in a different database . Such
information can be stored securely under the user's direct Alice (Wife ) ContactClass 1
Peter (Close Friend) ContactClass 2
control. The portal can retrieve information from the differ 10 Colina (Close Friend's wife ) ContactClass 2
ent database when required . In another embodiment, the user
can restrict or limit such retrieval process . Peter wants to make a mobile phone call to the user . In one
Additional confidentiality can be provided . In one embodiment, Peter calls a portal. As an example, the portal
embodiment, using phone calls as an example , the user can can be the user's ISP. The portal first verifies the caller's
beinformed
aware ofofthethenumber
identityof ofthethecallercaller even without being 15
. Similarly, the caller can
identity to be Peter. This can be done, for example, by a
public key challenge based on Peter having a public key
reach the user without being aware of the number of the digital certificate. In another example , Peter is also a regis
phone the user is using to receive the call . The user can keep tered user of the portal. Then , Peter's identity can be more
his location and /or status confidential but still can receive readily identified or verified .
the communication . This can be useful because there are 20 In one embodiment, after verification, a virtual address /
situations, for example , when the user does not want to number for the communication session is created allowing
disclose his contact information but the user needs to receive Peter to reach the user, which can be by phone. The user's
services provided by the caller. phone number does not have to be disclosed to Peter.
One approach to maintain such confidentiality while Similarly, Peter's mobile phone number does not have to be
maintaining real-time communication is based on a system 25 disclosed to the user. The portal can assure the user that the
that digitally identifies the identities of the caller and the person calling is Peter based on an identification verification
receiver. Note that the term caller is used in general. It is not process, such as ones described above.
just limited to phone calls, but they can be any person or In establishing contact, the portal can access the user's
entity requesting to communicate with the user, such as database and determine that Peter belongs to ContactClass2 .
trying to send a message to the user. As a separate note , the 30 The database can , for example , be in the portal.
caller can also be a user of different embodiments of the In another embodiment, the database is in a personal
invention . communication device of the user. The portal accesses the
After determining the identities , the system can establish personal communication device to determine Peter's Con
connections between the caller and the user in real time. tactClass .
Though contacts are established , the system only needs to 35 Based on the ContactClass information , the status of the
ensure the identities of the caller and the user to each other. user and Peter's urgency setting , the user may receive
However , the system does not have to disclose the phone Peter's call directly . As another example, Peter may be asked
numbers , electronic addresses, physical locations and /or to leave a voice mail to the user, while the user is notified by
other attributes of the caller and the user to each other. In one
a mobile short message regarding an incoming call from
embodiment, real time implies that the time required for the 40Peter.
identification is similar to the typical time required to set up ,As additional examples , in one embodiment, location
for example , a telephone call . The system can be a portal information of the user could be determined based on GPS
based on the web . information from , for example, the user's cell phone .
In one embodiment, a portal also holds the user's elec In one embodiment, the user receives messages through a
tronic calendar. The calendar can be programmable, with 45 handheld device , such as a phone, and the phone has a
entries set by the user . The portal can automatically and switch . The switch can be a physical button or a software
securely set appointments for the user since the portal knows setting , such as a pull -down menu . The user could set his
the identity of the caller, and the status and schedule of the status dynamically by changing the physical or logical
user. For example, the appointment can be for a conference position of the switch . For example , one position can
call. 50 indicate that the user is very busy, and should only be
To illustrate , in one embodiment, a portal provides a interrupted by an urgent message from the user's closest
number of intelligent communication modes (ICM ) for the contacts , such as his wife or parents . Another position can
user to select as shown in FIG . 1. There are three columns indicate that the user's status allows the user to receive any
in the table . If the communication mode selected in the messages from anyone .
second column does not work , the portal automatically 55 As explained above,based on an embodiment, a message
defaults to the corresponding approach in the third column. is electronically conveyed by a central network server , such
For example , under ICM 1, if the mobile phone is busy, as a web server based on Internet protocol. A portal or
default to voice mail. Someof the selections do nothave any gateway approach could provide general Internet access to
default because it may not be necessary to default. For one or more embodiments of the communication manage
example , under ICM 8 , the incoming message goes directly 60 ment systems so that users can configure the system behav
to voice mail with instant notification to mobile devices of ior they desire. The portal or gateway can then facilitate
the user. The incoming message can usually go to voice download of a database or update thereto to a communica
mail . There is no need to default. tion device , such as a phone.
As a receiver of communication , the user can define a Also , as explained above, based on an embodiment, a user
number of contact classes, as shown in FIG . 2. The user can 65 could efficiently maintain his communication , and does not
set up a number of urgency classes, as shown in FIG . 3. The even have to change phones when he moves from areas
user can define a number of status , as shown in FIG . 4. Then , covering 3G to areas that do not. These phones could be
US 10,708,727 B2
7 8
based on different communication mechanisms, such as prior history, time or other criteria . In one embodiment, the
GSM , CDMA , 36 and 4G systems. Also as explained above , communication between users is achieved by short audio or
the user could keep information in local databases, such as textualmessages.
in such a phone. For example, the intelligent communication The electronic device can be any computing device hav
modes shown in FIG . 1 for the user to select are in the phone. 5 ing communication capabilities. Such computing devices
The user could define the contact classes, such as the ones can be referred to as communication devices. Examples of
shown in FIG . 2 ; setup the urgency classes, such as the ones electronic devices include personal computers, personal
shown in FIG . 3 ; define the statuses , such as the ones shown digital assistants, pagers or mobile telephones.
in FIG . 4 ; set up the Access Priority Database , such as the 10 Embodiments of the invention are discussed below with
one shown in FIG . 5 ; and categorize a number of the user's reference to FIGS. 6-12 . However, those skilled in the art
contacts into the corresponding ContactClasses, all in the will readily appreciate that the detailed description given
phone . When a caller places a call to the phone , based on herein with respect to these figures is for explanatory
information previously set in the phone and based on the purposes as the invention extends beyond these limited
urgency class selected by the caller, the phone could auto 15 embodiments .
matically manage the communication . Note that the phone FIG . 6 is a communication system 100 according to one
does not have to be a cellular phone. In one embodiment,the embodiment of the invention . The communication system
phone is a desk top phone. 100 can support different communication devices, including
Again as explained above , the person or the caller trying mobile telephones 102 , computers 104 ( e.g., personal com
to contact the user could select different options. For 20 puters ) and /or wireless personal digital assistants (PDAs)
example , the urgency of the message can be set by the caller. 106. Users of the communication devices 102-106 can
This selection is typically in the call setup phase. In one communicate with like or different communication devices.
embodiment, the caller has pre-selected the urgency class Each communication device 102-106 offers one or both of
before making the call. In another embodiment, if the caller audio or textual communication capabilities. These commu
has not selected the urgency class , the system could prompt 25 nication devices 102-106 can inter-communicate with one
the caller to input an urgency class or status before the call another through a network 108. The network 108 can
or message is routed to the user. In yet another embodiment, include one or more of voice networks and data networks.
different urgency classes could be defined by the caller . For example , one network is a data network providing a
Different embodiments or implementations may yield slow speed data channel for transmission of ShortMessage
different advantages.One advantage is that communications 30 Service (SMS)messages (which are typically limited to 160
for users of electronic devices can be flexibly provided .
Another advantage is that communication mode changes can text characters) to a Short Message Service Center (SMSC )
be performed at an electronic device to better suit the needs and then forwarded on to the destination . Besides short
or condition of the electronic device or user preferences . In messages ( e.g., SMS messages ), the network 108 can also
still another advantage, a user can provide feedback to a 35 support other messaging protocols for sending and receiving
caller without answering a voice call from the caller. enhanced messages (EMS), multimedia messages (MMS),
Another aspect of the invention pertains to improved email and fax messages. Other networks support faster data
approaches to respond to an incoming text message. The channels and voice channels, such as GPRS, UMTS , G4,
improved approaches enable a recipient to provide a reply GSM , CDMA and various protocols, such as UDP, TCP ,
message to an initiator. The incoming text message can be 40 WAP , PDP other protocols.
presented to the recipient with an audio or textual presen According to one embodiment of the invention , one of the
tation . Thereafter, a reply text message can be sent back to communication devices 102-106 can send a shortmessage to
the initiator. The recipient can form the reply text message another of the communication devices 102-106 . The short
by recording a brief audio message or entering a text message can be text-based or audio -based . The sending
message . In the case in which a brief audio message is used , 45communication device allows its user to create the short
the audio message can be automatically converted to a text message as the user desires and /or as the device permits. For
message before being transmitted to the initiator. example , the user might interact with a keypad or keyboard
Further, the computer- implemented methods and systems to enter the short message , or the user might record audio
discussed above can be used in conjunction with one ormore inputs (e.g., speech ) for the short message . The short mes
of the various approaches discussed in U.S. patent applica- 50 sage can then be sent to the receiving communication
tion Ser. No. 11/006,343. For example , the automated device. The sending of the short message may involve
actions or decisions (e.g., intelligent secretary, decision 204converting the short message from an audio message to a
in FIG . 2 , etc.) of U.S. patent application Ser. No. 11/006 ,text message , or vice versa . Also , the receiving communi
343 can be automatically made by the systems/methods cation device can further convert the short message from
described above . Still further, the various approaches dis- 55 audio -to -text or from text-to -audio . In any case, the short
cussed in U.S. patent application Ser. No. 11/006,343 can be message is presented ( e.g., displayed or played ) to the user
used in conjunction with one or more the various methods/ of the receiving communication device . The presentation
systemsdiscussed above. For example, the systems/methods can vary as the user desires or as the device permits.
described above can use the messaging approaches (e.g., One aspect of the invention pertains to improved
audio or textual messages ) described in U.S. patent appli- 60 approaches to respond to incoming voice calls . The
cation Ser. No. 11/ 006,343 . improved approaches enable a called party ( i.e., a party
Different embodiments of the invention pertain to being called ) to provide some information to a calling party
improved approaches for users of electronic devices to without directly engaging in a voice call with the calling
communicate with one another. The electronic devices have party . The called party can choose not to take the voice call
audio and /or textual output capabilities. The improved 65 from the calling party. Instead , the called party can provide
approaches can enable users to communicate in different the calling party with some limited information . The limited
ways depending on device configuration , user preferences , information can be provided in an audio or textual format.
US 10,708,727 B2
9 10
In one embodiment, the limited information provides the sented to the user prior to the decision 204 of the personal
calling party with feedback as to why the voice call was not call response process 200 shown in FIG . 7 .
taken . In still another embodiment, an automated decision pro
FIG . 7 is a flow diagram of a personal call response cess to decide whether to answer a call can be based on time
process 200 according to one embodiment of the invention . 5 ( e.g., decision 204). For example, the called party can
The personal call response process 200 is performed by an previously set a rule , such as that from midnight to 6 am , the
electronic device , such as a mobile communication device party does not want to answer voice calls . Then , during this
(e.g.,mobile telephone ). The personal call response process not time period, the electronic device can automatically decide
200 begins with a decision 202 that determines whether to answer incoming calls . In one implementation , when
there is an incoming voice call. When the decision 202 10 notheindication
electronicofdevice
incomingdecides
callsnot
willtobeanswer
providedincoming calls,
to the called
determines that there is no incoming voice call , then the party . For example
personal call response process 200 awaits such a call. Once not produce any ring, from midnight to 6 am , the device would
the decision 202 determines that there is an incoming voice party can also configure .thetone Additionally , if desired , the called
electronic device to automati
call, a decision 204 determines whether the incoming voice 15 cally provide an audio message or a text message to the
call is to be answered . Typically, the user of the electronic calling party (e.g., I'm asleep call me tomorrow ” ).
device would signal the electronic device as to whether or FIG . 8 is a flow diagram of an audio message response
not to answer the incoming voice call . Alternatively , the process 300 according to one embodiment of the invention.
electronic device could automatically decide whether to The audio message response process 300 is , for example ,
answer the call. 20 suitable for use as the processing carried out by block 214
When the decision 204 determines that the user desires illustrated in FIG . 7 .
the incoming voice call to be answered , the incoming voice The audio message response process 300 initially answers
call is answered 206 and the user engages 208 in a voice call 302 the incoming voice call . In this operation , the incoming
with the calling party . A decision 210 then determines voice call is answered 302 but not in a traditional way .
whether the call has ended . When the decision 210 deter- 25 Instead , the electronic circuitry associated with a mobile
mines that the call has not yet ended , then the personal call communication device (e.g., mobile telephone) that receives
response process 200 can return to repeat the block 208 the incoming voice call operates to answer the incoming
while the voice call continues. Once the decision 210 voice call for purposes of an audio message response . For
determines that the voice call has ended , then the personal example , a voice channel is established between the calling
call response process 200 ends. 30 party and the mobile communication device , but the speaker
When the decision 204 determines that the user does not and microphone of the mobile communication device are
desire to answer the incoming voice call, a decision 212 disabled . In effect, in such an embodiment, neither the called
determines whether the user desires to provide an audio party nor the calling party perceives that the voice calling
message to the calling party. When the decision 212 deter has been answered .
mines that the user does desire to provide an audio message 35 Next, one or more predetermined audio messages can be
to the calling party , an audio message is obtained and sent presented 304 by the mobile communication device . The
214 to the calling party (caller ). presentation 304 of the one or more predetermined audio
Alternatively, when the decision 212 determines that the messages can , for example, be achieved by audio or visual
user does not desire to provide an audio message, a decision means. For example , the predetermined audio messages can
216 determines whether the user desires to provide a text 40 be audio output to a speaker associated with the mobile
message to the calling party . When the decision 216 deter communication device for the called party or can be visual
mines that the user desires to provide a textmessage to the output ( e.g., text) to a display of the mobile communication
calling party, a textmessage is obtained and sent 218 to the device for the called party ( e.g., user of the mobile commu
calling party. nication device ).
Still further, when the decision 216 determines that the 45 A decision 306 then determines whether a predetermined
user does not desire to provide a textmessage to the calling audio message has been selected . Here , the decision 306
party , a decision 220 determines whether the incoming voice determines whether the user ( i.e., called party ) of the mobile
call is to be directed to voice mail. When the decision 220 communication device has selected one or more of the
determines that the incoming voice call should be directed to predetermined audio messages . When the decision 306
voice mail, then the incoming voice call is directed 222 to 50 determines that a predetermined audio message has been
voice mail. On the other hand , when the decision 220 selected, then the selected audio message is played 308 for
determines that the incoming voice call is not to be directed the calling party. Here, the mobile communication device
to voice mail , the incoming voice call is dropped 224 . can output the selected audio message to the calling party
Following the blocks 214 , 218 , 222 and 224 , the personal over the voice channel. Typically, the mobile communica
call response process 200 is complete and ends. 55 tion device of the called party would not produce an audible
In another embodiment, a personal call response process output at the mobile communication device , so that the
could announce the calling party to the called party (user). called party would not be disturbed by the sending of the
In announcing the calling party, the personal call response audio response . The predetermined audio messages are
process would present the called party with information normally short messages (e.g., notmore than 160 characters )
pertaining to the calling party ( e.g., display or audio sound ). 60 so that the duration of time the voice channel is needed
Such information could, for example, help the called party and / or the amount of network bandwidth consumed is
to decide whether to answer the incoming voice call. The minimal.
information can , for example, include one or more of name On the other hand , when the decision 306 determines that
( individual or business ), telephone number , or other caller none of the predetermined audio messages have been
identification . The information could also include status 65 selected , then a decision 310 determines whether a custom
information of the calling party, such as position , health , audio message is requested . A custom audio message is an
mood , etc. As an example , the information could be pre audio message that is specifically provided for the calling
US 10,708,727 B2
11 12
party . When the decision 310 determines that a custom audio Next, one or more predetermined text messages are dis
message is not being requested , then the audio message played 404. Here , the one or more predetermined text
response process 300 returns to repeat the decision 306 and messages would normally be displayed on a display screen
subsequent operations. Alternatively , when the decision 310 associated with the mobile communication device being
determines that a custom audio message is requested , then a 5 utilized by the called party . A decision 406 then determines
custom audio message is recorded 312. Thereafter, the has whether one (or more ) of the predetermined text messages
custom audio message that has been recorded can be played predetermined been selected . When the decision 406 determines that a
314 for the calling party ( caller ). Here , typically, the custom text message has been selected , then the
audio message would be output by the mobile communica 10 the calling party ). is transmitted 408 to the caller ( i.e.,
selected text message
tion device of the called party over the voice channel to the On the other hand, when the decision 406 determines that
calling party . Typically, themobile communication device of a predetermined textmessage has not been selected , then a
the called party would not produce an audible output at the decision 410 determines whether a custom text message is
mobile communication device, so that the called party requested . When the decision 410 determines that a custom
would notbe disturbed by the sending of the audio response. 15 text message is not requested, then the text message
The custom audio messages are also normally short mes response process 400 returns to repeat the decision 406 and
sages (e.g., not more than 160 characters ) so that the subsequent operations . Alternatively , when the decision 410
duration of time the voice channel is needed and / or the determines that a custom text message is requested , then the
amount of network bandwidth consumed is minimal. custom text message is entered 412. Here, the called party
Following the operations 308 and 314 , the incoming voice 20 interacts with the mobile communication device to enter the
call is closed 316. In other words, after the selected audio custom text message. Then , the custom text message is
message or the custom audio message is played 308 , 314 , the transmitted 414 to the caller. In one embodiment, the trans
incoming voice call can be closed 316. Following the block mission 408,414 of the textmessage can be performed over
316 , the audio message response process 300 is complete a communication network , such as a network having a Short
and ends. 25 Message Service Center (SMSC ) supporting Short Message
The predetermined audio messages that are presented 304 Service (SMS) messages. Following the transmission 408 of
to a called party can be determined in a static or dynamic the selected text message or the transmission 414 of the
manner. A static determination would , for example , be when custom text message, the textmessage response process 400
the called party has previously set or recorded an audio is complete and ends .
message to be utilized . Typically , with static determination , 30 An alternative embodiment of a text message response
the list of audio messages remains the same ( i.e., static ) until process could operate to answer the incoming voice call and
changed ( e.g., by the called party ). A dynamic determination announce to the caller that a text message will be forthcom
would allow the audio messages in the list (or the ordering ing . Then , the incoming voice call could be promptly
of the audio messages in the list ) to change without specific dropped . This additional operation could , for example , be
action by the user or the called party . For example , the list 35 used with the text message response process 400 by pro
or ordering of the audio messages can depend on preference viding an additional operation prior to the block 402 illus
settings, configuration information , or prior usage . Prior trated in FIG . 9 .
usage can include biasing the list of audio messages such The predetermined textmessages being displayed 404 to
that those messages being most often selected appear higher a called party can be determined in a static or dynamic
in the list. The list or ordering of the audio messages can also 40 manner. A static determination would , for example, be a text
depend on the calling party, type of calling party, location of message the called party has previously set or entered .
calling party or called party , and the like . The list of audio Typically, with static determination , the list of text messages
messages can be represented by text and /or graphics (e.g., remains the same ( i.e., static ) until changed ( e.g., by the
icons) called party ). A dynamic determination would allow the text
The audio message response process 300 flexibly enables 45 messages in the list ( or the ordering of the text messages in
a user to either select one or more predetermined audio the list) to change automatically, and not by the user. For
messages or provide a custom audio message to be used as example , the list or ordering of the text messages can depend
an audio message response to a calling party. However , it on preference settings, configuration information , or prior
should be recognized that, in other embodiments , an audio usage. To illustrate, prior usage can include biasing the list
message response process can alternatively simply pertain to 50 of text messages such that those messages being mostoften
only providing a custom audio message , or only permitting selected appear higher in the list. The list or ordering of the
selection of a predetermined audio message. Further, in still text messages can also depend on the calling party, type of
other embodiments, an audio message response process can calling party, location of calling party or called party, and the
first determine whether a custom audio message is to be like . The list of text messages can identify each textmessage
provided before presenting predetermined audio messages . 55 with text ( e.g., at least a portion of the corresponding text
In yet other embodiments, an audio message response pro message, or an abbreviation ) and /or graphics (e.g., icons).
cess can answer the incoming voice call later in the pro The text message response process 400 flexibly enables a
cessing than operation 302 as shown in FIG . 8 (e.g., before user to either select one or more predetermined text mes
operations 308 and 314 ). sages or provide a custom textmessage to be used as a text
FIG . 9 is a flow diagram of a text message response 60 message response to a calling party . However, it should be
process 400 according to one embodiment of the invention . recognized that, in other embodiments, a text message
The text message response process 400 is, for example , response process can alternatively simply pertain to only
processing performed by the block 218 illustrated in FIG . 7 . providing a custom text message, or only permitting selec
The textmessage response process 400 initially drops 402 tion of a predetermined textmessage . Further, in still other
the incoming voice call.Here, the information to be supplied
to the calling party is a short textmessage; therefore , there
65 embodiments , a text message response process can first
determine whether a custom text message is to be provided
is no need for a voice channel. before presenting predetermined textmessages.
US 10,708,727 B2
15 16
The message presentation process 600 begins with a server coupled to the network 108. One example of a
decision 602 that determines whether an incoming text networked server is a gateway computer for a wireless
message is present. Typically, the incoming text message electronic device , such as a mobile telephone .
would be transmitted to the mobile communication device FIG . 12 is a flow diagram of a reply message process 700
from another communication device. When the decision 602 5 according to one embodiment of the invention . The reply
determines that an incoming text message is not present, message process 700 is performed by an electronic device,
then the message presentation process 600 awaits such such as a mobile communication device .
message . Once the decision 602 determines that an incom The reply message process 700 begins with a decision 702
ing text message has been received , a decision 604 deter that determines whether a reply message is to be sent.
mines whether an audio or text presentation is to be utilized . 10 Typically , the reply message process 700 follows the pre
The decision 604 can be performed in a variety of different sentation of an incoming textmessage to a user of a mobile
ways . For example , the determination of whether to utilize communication device . Hence, the reply message to be sent
an audio or text presentation can be based on user input or is a reply to the incoming text message. However, in other
can be automatically determined through a use of configu embodiments, the reply message to be sent can be merely an
ration or preference information or hardware components 15 message
(e.g., display, speaker, head - set).
initial message
.
as opposed to a response to an earlier
When the decision 604 determines that an audio presen In any case , when the decision 702 determines that a reply
tation is to be utilized , the incoming text message is con message is not to be sent, then the reply message process 700
verted 606 to an audio message. For example , a text-to ends or simply awaits the need to send a reply message. On
speech conversion can be performed . In one embodiment, a 20 the other hand, when the decision 702 determines that a
user of the electronic device can be permitted to choose reply message is to be sent, then a decision 704 determines
speech characteristics, such as a voice, tone, pace, accent, or whether an audio or text message is to be formed . The
mood , for the resulting speech . For example, a user could decision 704 can be performed in a variety of different ways .
choose speech characteristics by preference settings. In For example , the determination of whether to send an audio
another embodiment, the incoming text message can include 25 or text message can be based on user input or can be
or reference speech characteristics so that the initiator can automatically determined through a use of configuration or
control or influence speech characteristics . In still another preference information or hardware components (e.g., dis
embodiment, if the text to be converted contains condensed play, speaker, head -set).
text (e.g., such as slang or chat language ), the resulting When the decision 704 determines that an audio message
speech can pertain to an uncondensed form of the text. The 30 is to be formed , then the reply message process 700 prompts
ability to convert from condensed text to resulting speech for 706 for an audio message. Here , the prompt 706 can be
uncondensed text can be facilitated by pattern matching. For directed to the user of the mobile communication device .
example , in chat language “ LOL ” can be converted to an The prompt can be an audio or textual indication. Next, a
audio message for lots of love ." In one implementation , a decision 708 determines whether an audio message has been
table can store audio messages corresponding to chat terms 35 recorded. When the decision 708 determines that the audio
or phrases . In another implementation , a first table would message has not been recorded , then the reply message
store uncompressed terms or phrases corresponding to chat process 700 awaits the audio message . Once the decision
terms or phrases, and a second table would store audio 708 determines that the audio message has been recorded ,
messages corresponding to the uncompressed terms or then the audio message is converted 710 to a text message .
phrases. 40 In one embodiment, if the audio message recorded is greater
After the incoming text message is converted to the audio than a maximum text message size (e.g., 150 or 160 char
message , the audio message is played 608. Typically , the acters), then the audio message can be shortened so that the
audio message is played 608 by the mobile communication resulting text message does not exceed the maximum text
device for the user. For example, the audio message can be message size . One way to shorten the text message is to use
output to a speaker of the mobile communication device or 45 abbreviations . For example, the words“ For example” can be
a headset used therewith . As a result , the user of the mobile changed to “ e.g.” . Such conversion can be again be per
wireless communication device receives an audio message formed by matching entries in tables. Another way to
even though the incoming message was a text message . shorten is to remove non - essential text. Still another way to
On the other hand , when the decision 604 determines that shorten is to clip off or truncate the text message at the
a text presentation is to be utilized , the incoming text 50 maximum text message size. In another embodiment, the
message is displayed 610. Here, the incoming textmessage resulting text message might provide an indication that it
would be displayed 610 on a display associated with the was converted from an audio message. Following the block
mobile communication device. Following the blocks 608 710 , the text message is transmitted 712 over a wireless
and 610 , the message presentation process 600 ends. network .
As discussed above, text-to -speech conversion can be 55 Alternatively, when the decision 704 determines that a
invoked and performed on an electronic device , which may text message is to be formed , then a text entry screen is
be a mobile communication device . While text- to -speech displayed 714. Next, a decision 716 determines whether a
conversion , particularly if high quality is desired , requires text message has been entered . When the decision 716
substantial processing capabilities, mobile electronic determines that a textmessage has not yet been entered , then
devices, such as mobile communication devices, given their 60 the reply message process 700 awaits entry of the text
small form factor and price competition , tend to have limited message. Once the text message has been entered , the text
processing capability . Accordingly, in one embodiment, message is transmitted 712 over the wireless network .
text-to -speech conversion can be off -loaded from themobile Following the block 712 , the reply message process 700
device . For example , a remote server computer can be ends.
provided the text message and produce the resulting audio 65 Although the reply message process 700 provides for the
message, and then supply the audio message to the mobile user to enter a custom text or audio message, it should be
device . The remote server computer can be a networked understood that the reply message can alternatively be
US 10,708,727 B2
17 18
formed through use of semi-custom or predetermined reply tage is that the sender can get back a message in the same
messages from which the user of the mobile communication format as they sent the originalmessage , though the recipi
device can choose . The use of semi-custom or predeter ent may use the message in a different format or mode ( e.g.,
mined reply messages can be achieved as noted above in a recipient hears the textmessage as an audio message).
number of embodiments, and can serve to simplify the 5 Moreover, it should be noted that with regards to any of
conversion process . the embodiments in which a voice call or a text message is
An exemplary scenario of how message presentation and incoming to an electronic device , notonly can the user of the
reply message processes could work according to one imple mobile device take an action ( e.g., button press or voice
mentation of the second aspect is as follows: command ) to decline the call/message but also the electronic
1. From his mobile phone , Bill prepares and sends a text 10 device itself can automatically decline the call /message such
message to Tom's mobile phone . that the user is not disturbed . For example, an electronic
2. Tom is alerted by his mobile phone of an incoming text device can be configured through user settings ( e.g., pref
message , such as by displaying at least a portion of the text erences) to decline calls/messages matching certain criteria .
message and/or otherwise notifying Tom of the text mes Also , an auto reply message can be configured to be auto
sage . 15 matically sent in response to the call /message. For a known,
3. Tom's mobile phone can decide whether to present the undesired marketing caller/message sender, the electronic
textmessage on a display screen of Tom's mobile phone, or device can automatically send a reply message demanding
to first convert the text message to an audio message and the sender not to call or send messages anymore , and to
then present the audio message to Tom (e.g., play the audio remove your information from their database .
message). Of course , Tom can interact with Tom's mobile 20 Text messages received or sent can optionally embed
phone to assist in making the determination on how to indications of speech characteristics to be used , should the
present the message. text message be converted to an audio format. The speech
4. Thereafter , if desired , Tom can prepare and send a reply characteristics can pertain to voice , tone, pace , accent,
message back to Bill . This reply message can be prepared and /or mood . The speech characteristics for the resulting
initially as a text message or an audio message. Tom's 25 speech can be set in preference or configuration information,
mobile phone and/or Tom can determine whether the reply set on a per message basis by users, or set by evaluation of
message is initially prepared as a textmessage or as an audio monitored data pertaining to the user.
message . If an audio message is initially created , such audio Additionally, the messages being transmitted can be
message must be converted to a text message prior to encrypted for security purposes .
transmission . Eventually, the reply message is sent to Bill as 30 In one embodiment, an electronic device performing
a text message . Tom's mobile phone can assist with the communications using audio and /or text messages according
creation of the reply message through use of custom , semi to the invention can further integrate (or have tethered
custom or predetermined reply message from which Tom thereto ) one or more electrical components for enhancing
and / or Tom's mobile phone can choose . the hearing of the user of the electronic device. The elec
5. If Tom is using a head -set with his mobile phone, then 35 tronic device will normally include a microphone and a
an audio message may be more convenient, assuming that speaker. The invention described herein can be considered
Tom wants to provide a particular ( i.e. , customized )message an automated secretary for a user of an electronic device .
to Bill . The head -set allows Tom to easily record a brief The automated secretary can completely or partially respond
audio message. Less conveniently , the head -set can be used to an incoming call/message so as to reduce disturbances to
to present a list of predetermined audio messages and allow 40 the user. The user can personalize the automated secretary
Tom's selection therefrom by a button or voice -command . through user settings ( e.g., preferences ), or the automated
6. If Tom is not using a head - set, then a text message secretary can learn over time how to handle different incom
response might be more suitable. This would typically ing calls/messages. Besides handling or assisting the user
require that Tom's mobile phone have a display and a with incoming calls/messages , the automated secretary can
keypad . Even so , without a head -set , Tom could still record 45 also assist with other activities , such as making calendar
an audio message, though such would likely be less conve entries (e.g., meetings) in a calendar or responding to
nient. incoming callers /messages with relevant information per
7. Tom can also not provide a reply message and simply taining to the user's schedule asmaintained by the calendar
not respond to the incoming text message . Alternatively, ( though the user could restrict such access to certain infor
Tom can configure his mobile phone to automatically pro- 50 mation and/or inquiring parties ). For example , if an incom
duce and send a reply message based on user settings or ing text message asks “ available for lunch today ?” , the
preferences, position , configuration , status , etc. automated secretary can check the user's availability for
In this aspect of the invention , the calling party and the lunch by way of the user's calendar, then if the user is not
called party often use mobile communication devices, such available the automated secretary can quickly informing the
as mobile phones. However, the parties can alternatively use 55 inquiring party of same or propose another date . On the
other electronic devices, such as a PDA , a computer, etc. other hand, if the lunch time period is available in the user's
The advantages of the invention exemplified by FIGS. calendar , then the automated secretary can either directly
11-12 are numerous. Different embodiments or implemen respond to the inquiring party of acceptance or propose a
tationsmay yield different advantages. One advantage is that response to the user for review , modification and /or trans
communications for users of electronic devices can be 60 mission .
flexibly provided . Another advantage is that communication It should be obvious to those skilled in the art that a
mode changes can be performed at an electronic device to number of embodiments performing communications using
better suit the needs or condition of the electronic device or voice as well as audio and /or text messages can be imple
user preferences. Still another advantage is that conversion mented using voice over Internet Protocol technologies,
of an audio message to a textmessage facilitates use a low 65 with signals delivered over the Web . For example, a calling
cost network ( such as the SMS network ). Another advantage party's communication or mobile device can include an
is reduced network bandwidth load . Yet still another advan adapter to convert voice signals to data packets before
US 10,708,727 B2
19 20
sending them over the Internet. A service provider can munication for a first message to be sent from the first
convert the packets back into voice signals before sending user to a second user, based on an identifier associated
the voice signals to the called party's communication with the first user previously set by the first user via the
device. Similarly, embodiments can be implemented using network -based portal,
voice over wireless protocols , such as Wi-Fi or Wi-Max 5 wherein the plurality ofmodes of communication sup
networks. Using such technologies, computing devices can ported by the network -based portal include at least
become communication devices. text communication using a personal computer,
The various embodiments, implementations , features and voice communication using a personal computer ,
aspects of the invention noted above can be combined in and communication with at least an image, and
various ways or used separately . Those skilled in the art will 10 wherein messages are eligible to be received by the
understand from the description that the invention can be second user via the network - based portal, based on
equally applied to or used in other different settings with any of the plurality of modes of communication , all
respect to various combinations, embodiments , implemen depending on an identifier associated with the second
tations or features provided in the description herein . user previously set by the second user via the net
The invention can be implemented in software , hardware 15
or a combination of hardware and software . A number of work -based portal, which allows the second user to
embodiments of the invention can also be embodied as efficiently maintain the second user's communica
computer readable code on a computer readable medium . tion using the plurality ofmodes of communication ;
The computer readable medium is any data storage device enabling the second user to block the first user from using
that can store data which can thereafter be read by a 20 at least the selected mode of communication to com
computer system . Examples of the computer readable municate with the second user via the network -based
medium include read -only memory, random - access portal, based on the identifier associated with the first
memory, CD -ROMs, magnetic tape, optical data storage user ;
devices, and carrier waves. The computer readable medium enabling the first message to be provided to the second
can also be distributed over network -coupled computer 25 user, using the selected mode of communication ,
systems so that the computer readable code is stored and depending on the identifier associated with the second
executed in a distributed fashion . user, in view of the second user not blocking the first
Numerous specific details are set forth in order to provide user from using the selected mode of communication to
a thorough understanding of the present invention .However, communicate with the second user, via the network
it will become obvious to those skilled in the art that the 30 based portal;
invention may be practiced without these specific details . and
The description and representation herein are the common receiving a second message from the second user to the
meanings used by those experienced or skilled in the art to first user, to respond to the first message , after the
most effectively convey the substance of their work to others second user has received the first message ,
skilled in the art. In other instances, well -known methods, 35 wherein one of the first message or the second message is
procedures, components, and circuitry have not been voice and the other of the first message or the second
described in detail to avoid unnecessarily obscuring aspects message is text,
of the present invention .
In the foregoing description , reference to " one embodi wherein even when the first message is received by the
ment” or “ an embodiment” means that a particular feature, 40 second user via the selected mode of communication ,
structure, or characteristic described in connection with the contact information associated with the second user and
embodiment can be included in at least one embodiment of provided by the second user to the network -based
the invention . The appearances of the phrase “ in one portal is not provided via the network -based portal to
embodiment” in various places in the specification are not the first user, and contact information associated with
necessarily all referring to the same embodiment, nor are 45 the first user and provided by the first user to the
separate or alternative embodiments mutually exclusive of network -based portal is not provided via the network
other embodiments . Further, the order of blocks in process based portal to the second user, so as to provide an
flowcharts or diagrams representing one or more embodi option to the second user to keep the contact informa
ments of the invention do not inherently indicate any par tion associated with the second user confidential from
ticular order nor imply any limitations in the invention . 50 the first user, and to provide an option to the first user
The many features and advantages of the present inven to keep the contact information associated with the first
tion are apparent from the written description and, thus, it is user confidential from the second user,
intended by the appended claims to cover all such features wherein the identifier associated with the second user is
and advantages of the invention . Further, since numerous distinct from the contact information associated with
modifications and changes will readily occur to those skilled 55 the second user, and the identifier associated with the
in the art, it is not desired to limit the invention to the exact first user is distinct from the contact information asso
construction and operation as illustrated and described . ciated with the first user,
Hence , all suitable modifications and equivalents may be wherein the contact information associated with the sec
resorted to as falling within the scope of the invention . ond user includes at least one of a phone number or an
What is claimed is : 60 email address of the second user, and
1. A computer- implemented method to facilitate elec wherein the contact information associated with the first
tronic communication of a plurality of users using at least a user includes at leastone of a phone number or an email
network -based portal at least based on Internet protocol, the address of the first user.
method comprising: 2. A computer- implemented method as recited in claim 1
providing a plurality ofmodes of communication to a first 65 comprising enabling a message to be provided to the second
user to allow the first user to use one of the plurality of user from the first user, with the message based on a mode
modes of communication as a selected mode of com of communication that is different from the selected mode,
US 10,708,727 B2
21 22
even though the second user has blocked the first user from 10. A computer-implemented method as recited in claim
using the selected mode of communication to communicate 1,
with the second user . wherein the first message is a text message, and
3. A computer -implemented method as recited in claim 2 , wherein the method comprises transforming the first
wherein the mode of communication different from the 5 message into a voice message so that the first message
selected mode includes communication with at least an is provided to the second user in an audio manner via
image and communication in an audio manner . a speaker if the first user is not blocked from using the
4. A computer- implemented method as recited in claim 2 selected mode of communication to communicate with
comprising enabling a message to be provided to the second the second user .
user from another user using the selected mode to commu- 10 11. A computer-implemented method as recited in claim
nicate, but not enabling messages to be provided to the 10, wherein the method comprises transforming the second
second user from the another user using the mode of message from voice into text to be provided to the first user.
communication different from the selected mode to com 12. A computer -implemented method as recited in claim
municate , via the network -based portal. 1 , wherein the method comprises:
5. A computer- implemented method as recited in claim 1 15 keeping track of the number
comprising : of times the second user has
enabling a message to be provided to the second user from blocked the first user to communicate with the second
the first user, with the message based on a mode of user; and
communication that is different from the selected blocking the first user to communicate with the second
mode; and 20 user at least via the selected mode of communication
enabling amessage to be provided to the second user from after the number of times has reached a preset number .
another user using the selected mode to communica 13. A computer-implemented method as recited in claim
tion , but not enabling messages to be provided to the 12 , wherein the method comprises asking the second user for
second user from the another user using the mode of permission before blocking the first user to communicate
communication different from the selected mode , via 25 with the second user at least via the selected mode of
the network -based portal. communication .
6. A computer - implemented method as recited in claim 1 , 14. A computer- implemented method as recited in claim
wherein the plurality ofmodes of communication supported 1, wherein the method comprises receiving at least an
by the network -based portal include at least communication indication from the second user specifying the first user
using
whereinan electronic
the methoddevice withouta user
supports a computer
to use keyboard , and 30 having a rightto access the second user beforemessages are
an electronic
device with a computer keyboard to communicate with eligible via the
to be provided to the second user from the first user
network -based portal.
another user using an electronic device without a computer 15. A computer -implemented method as recited in claim
keyboard .
7. A computer-implemented method as recited in claim 1 35 1, wherein the method
determining comprises
availability of the: second user related to
comprising : receiving messages; and
receiving an indication from the second user for the first sending data to the first user regarding the availability of
user via the network -based portal to generate an urgent the second user related to receiving messages .
notification for the first user; and 16. A computer-implemented method as recited in claim
enabling the urgent notification to be generated to notify 40 1 comprising
the first user at least in view of receiving the indication . mined messageenabling the second user to select a predeter
8. A computer -implemented method as recited in claim 7 , message being selected tofromthe afirst
to send user, the predetermined
wherein the urgent notification is enabled to be generated at sages provided to the second user. of predetermined mes
set
least in an audio manner.
17. A computer-implemented method as recited in claim
9. A computer -implemented method as recited in claim 1 45 1 wherein
comprising enabling an urgent notification to be provided to the plurality of communication modes include at
the second user from the first user, even though the second least text communication using a mobile phone , and voice
user has blocked the first user from using the selected mode communication using a mobile phone.
of communication to communicate with the second user.
EXHIBIT B
US010492038B2
(12) United States Patent

Cheung et al.
( 10 ) Patent No.: US 10,492,038 B2
(45 ) Date of Patent: *Nov. 26 , 2019
(54 ) METHOD AND APPARATUS TO MANAGE (58 ) Field of Classification Search
MESSAGING PROVIDING DIFFERENT ??? H04M 2203/651; H04M 3/42059; H04M
COMMUNICATION MODES DEPENDING ON 3/42093
ONE IDENTIFIER AND NOT REQUIRING TO (Continued )
DISCLOSE CONTACT INFORMATION
(56 ) References Cited
( 71 ) Applicant: Ip Venture , Inc. , Los Altos, CA (US) U.S. PATENT DOCUMENTS
(72 ) Inventors: Kwok Wai Cheung, Tai Po (HK ); 5,425,516 A 6/1995 Daines
Peter P. Tong, Mountain View , CA 5,548,636 A 8/1996 Bannister et al.
(US) ; C. Douglass Thomas , Saratoga , (Continued )
CA (US)
FOREIGN PATENT DOCUMENTS
(73 ) Assignee : Ip Venture, Inc., San Jose , CA (US )
CN 1453981 11/2003
( * ) Notice : Subject to any disclaimer, the term of this WO WO 01/45343 A2 6/2001
patent is extended or adjusted under 35
U.S.C. 154 (b ) by 0 days. OTHER PUBLICATIONS
This patent is subject to a terminal dis Notice of Allowance for U.S. Appl. No. 12 /798,995 dated Sep. 29 ,
claimer. 2011.
(21) Appl. No.: 15 /704,181 (Continued )
(22) Filed : Sep. 14 , 2017 Primary Examiner — Kiet M Doan
(65 ) Prior Publication Data (57) ABSTRACT
US 2018/0014169 A1 Jan. 11, 2018 A number of embodiments regarding computer-imple
mented systems and methods to manage electronic commu
nications are disclosed . In one embodiment, an apparatus,
Related U.S. Application Data using at least a network -based portal based on Internet
(63 ) Continuation of application No. 15/ 469,440, filed on protocol, provides different communication modes to a first
Mar. 24 , 2017 , now Pat. No. 10,142,810 , which is a user, with messages eligible to be received by a second user
(Continued ) based on any of the modes, depending on an identifier
associated with the second user ; and receives an indication
(51) Int. Ci. from the first user to select a communication mode for a
H04W 4/14 (2009.01) message for the second user. In the embodiment, the appa
G06Q 10/10 (2012.01) ratus could receive contact information associated with the
(Continued ) second user to allow the second user to participate , with the
(52) U.S. Cl. contact information not provided to the first user even when
CPC
the message is received by the second user, and with the
H04W 4/14 (2013.01); G06Q 10/10 contact information being distinct from the identifier.
(2013.01) ; H04M 3/436 (2013.01);
(Continued ) 70 Claims, 9 Drawing Sheets
START 200
2024
INCOMING
VOICE NO
CALL
2
YES
204
206
YES
ANSWER
ANSWER THE NO
INCOMING VOICE 212
CALL
AUDIO
MESSAGE 214
YES
ENGAGE IN
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 216
NO 7
TEXT
MESSAGE YES 218
YES
OBTAIN AND SEND
NO TEXT MESSAGE
TO CALLER
220
VOICE 222
MAIL YES
DIRECT TO
NO VOICE MAIL
DROP THE INCOMING 224
VOICE CALL
END
US 10,492,038 B2
Page 2
Related U.S. Application Data 7,686,693 B2 3/2010 Danieli et al.

6/2010 Cheung et al.
7,729,688 B2
continuation of application No. 14 /922,344 , filed on 7,792,552 B2 9/2010 Thomas et al.
Oct. 26 , 2015 , now Pat. No. 9,736,664 , which is a 7,890,128 B1 2/2011 Thomas et al.
continuation of application No. 14 /272,632 , filed on 8,112,104 B1 2/2012 Thomas et al.
8,280,419 B1 10/2012 Thomas et al.
May 8 , 2014 , now Pat. No. 9,204,268, which is a 8,353,773 B2 1/2013 Sasaki et al.
continuation of application No. 12 /798,995 , filed on 8,429,231 B2 4/2013 Wu et al.
Apr. 14 , 2010 , now Pat . No. 8,744,407 , which is a 8,391,459 B2 * 5/2013 Jasckson et al. G06Q 10/109
continuation of application No. 11/ 452,115 , filed on 379 /210.01
Jun . 12 , 2006 , now Pat. No. 7,729,688, which is a 8,737,978 B1 5/2014 Thomas et al.
continuation - in -part of application No. 11/006,343 , 8,744,407 B2 6/2014 Cheung et al.
8,827,811 B2 9/2014 Kim et al.
filed on Dec. 7 , 2004 , now Pat. No. 7,116,976 . 9,204,268 B2 12/2015 Cheung et al.
9,555,334 B2 1/2017 Bernard et al.
(60 ) Provisional application No.60/527,565, filed on Dec. 9,736,664 B2 8/2017 Cheung et al.
8 , 2003, provisional application No. 60 /689,686 , filed 10,142,810 B2 11/2018 Cheung et al.
on Jun . 10 , 2005 . 10,183,219 B2 1/2019 Linden et al.
10,207,191 B2 2/2019 Jensen
2001/0011014 A1 8/2001 Higuchi et al.
(51) Int. Cl. 2001/0014611 Al 8/2001 Dufort
H04M 3/436 ( 2006.01) 2001/0028709 Al 10/2001 Makela et al.
H04W 4/16 ( 2009.01) 2001/0031633 Al 10/2001 Tuomela et al.
H04W 4/12 ( 2009.01) 2002/0067806 Al 6/2002 Rodriguez et al.
2002/0073207 A1 6/2002 Widger et al.
(52 ) U.S. Ci. 2002/0094067 A1 7/2002 August
CPC H04M 3/4365 (2013.01); H04W 4/12 2002/0142756 Al 10/2002 Rutledge et al.
2002/0181672 A1 12/2002 Cannell et al.
(2013.01) ; H04W 4/16 ( 2013.01); H04M 2003/0039339 A1 2/2003 Luehrig et al.
2201/60 (2013.01 ); H04M 2203/2011 2003/0041048 A1 2/2003 Balasuriya
(2013.01) ; H04M 2203/651 ( 2013.01); H04M 2003/0065779 Al 4/2003 Malik
2207/18 (2013.01) 2003/0103600
2003/0105854
A1
A1
6/2003
6/2003
Potter
Thorsteinsson et al .
(58 ) Field of Classification Search 2003/0112948 A1 6/2003 Brown et al .
USPC 455/417 , 414.1 , 466 , 412.2 , 435.3 ; 2003/0129968 Al 7/2003 Earl
379 /88.16 , 390.01, 207.04 2003/0191676 A1 10/2003 Templeton
See application file for complete search history . 2003/0191814 A1 10/2003 Tran
2003/0232629 Al 12/2003 Jang et al.
( 56 ) References Cited 2004/0024882 A1 2/2004 Austin et al .
2004/0072585 Al 4/2004 Le et al.
U.S. PATENT DOCUMENTS 2004/0078340 A1 * 4/2004 Evans G06Q 20/10
705/64
5,610,970 A 3/1997 Fuller et al . 2004/0122979 A1 6/2004 Kirkland
5,752,191 A 5/1998 Fuller et al . 2004/0143667 A1 * 7/2004 Jerome G06F 17/30882
5,758,079 A 5/1998 Ludwig et al. 709/228
5,828,731 A 10/1998 Szlam et al. 2004/0203919 Al 10/2004 Ross et al.
5,930,700 A * 7/1999 Pepper et al. H04Q 7/24 2004/0240650 Al 12/2004 Bear et al.
455/414 2004/0248596 Al 12/2004 Panchal
5,970,388 A 10/1999 Will 2005/0020288 Al 1/2005 Davis
6,119,022 A 9/2000 Osborn et al. 2005/0027385 A1 2/2005 Yueh
6,327,628 B1 12/2001 Anuff et al . 2005/0037785 Al 2/2005 Chen
6,359,982 B1 3/2002 Szlam 2005/0038690 A1 2/2005 Hayes-Roth
6,463,462 B1 10/2002 Smith et al. 2005/0071253 A1 3/2005 Yang
6,577,859 B1 6/2003 Zahavi et al. 2005/0107130 A1 5/2005 Peterson , II
6,636,888 B1 10/2003 Bookspan et al. 2005/0136955 A1 6/2005 Mumick et al.
6,665,534 B1 12/2003 Conklin et al.
6,788,766 B2 9/2004 Logan 2005/0191994 Al 9/2005 May et al.
6,801,793 B1 10/2004 Aarnio et al . 2005/0192061 A1 9/2005 May et al.
6,816,578 B1 11/2004 Kredo et al . 2005/0273327 Al 12/2005 Krishnan
6,819,757 B1 11/2004 Cook et al. 2006/0003803 Al 1/2006 Thomas et al.
6,819,945 B1 11/2004 Chow et al. 2006/0075038 Al 4/2006 Mason et al.
6,978,136 B2 12/2005 Jenniges et al. 2006/0168054 A1 * 7/2006 Burkhart H04L 51/04
7,010,288 B2 3/2006 Brown et al. 709/206
7,010,332 B1 3/2006 Irvin et al . 2006/0212561 A1 * 9/2006 Feng HO4L 63/0227
7,027,842 B2 4/2006 Zhang et al. 709/223
7,043,261 B2 5/2006 Krishnan 2006/0239419 Al 10/2006 Joseph et al.
7,072,452 B1 7/2006 Roberts et al. 2006/0259565 A1 11/2006 Cheung et al .
7,085,253 B2 8/2006 Yang 2006/0276210 A1 12/2006 Thomas et al.
7,107,010 B2 9/2006 Heinonen et al. 2006/0288099 A1 12/2006 Jefferson et al.
7,110,963 B2 9/2006 Negreiro 2007/0005368 A1 1/2007 Chutorash et al.
7,111,044 B2 9/2006 Lee
7,116,976 B2 10/2006 Thomas et al. 2007/0047522 A1 3/2007 Jefferson et al.
7,188,073 B1 3/2007 Tam et al. 2007/0238474 Al 10/2007 Ballas
7,224,775 B1 5/2007 Shaffer et al. 2008/0261636 A1 10/2008 Lau et al.
7,317,706 B1 1/2008 Hao et al . 2010/0114958 Al 5/2010 Korenshtein
7,346,630 B2 3/2008 Eichstaedt et al. 2010/0205272 Al 8/2010 Cheung et al.
7,376,434 B2 5/2008 Thomas et al. 2011/0151582 Al 6/2011 Basile
7,376,630 B2 5/2008 Blair et al. 2011/0151852 Al 6/2011 Olincy et al.
7,403,972 B1 7/2008 Lau et al. 2014/0242956 A1 8/2014 Cheung et al.
US 10,492,038 B2
Page 3
( 56 ) References Cited BlackBerry, “ Voice and SMS”, http://www.blackberry.com/products/

service /voices_sms.shtml?DCPID = hmsvoice downloaded Oct. 2 ,
U.S. PATENT DOCUMENTS 2003, p . 1.
2014/0256293 Al 9/2014 Thomas et al.
Calsyn ,Martin and Desseault , Lisa, “ Presence Information Protocol
Requirements,” Internet Draft, Feb. 9 , 1998, pp . 1-27.
2016/0044474 Al 2/2016 Cheung et al. Emergin Inc., “ Emergin WirelessOffice 5.0 ” , http: //www.emergin .
2017/0201872 A1 7/2017 Cheung et al. com /? source = overture , downloaded Oct. 2 , 2003 , p . 1.
Fastmobile Inc., “Dialog GSM launches Push ‘n ’ Talk walkie talkie
OTHER PUBLICATIONS service Push to Talk over Cellular Now in Sri Lanka Dialog GSM
Pioneers Latest GSM Advancement” , Press Release , Dec. 1, 2004 ,
Notice of Allowance for U.S. Appl. No. 12/ 798,995 dated Jan. 3 , pp . 1-2 .
2012 . Fastmobile, “ fastmobile's fastchatTM Instant Communications Appli
Notice of Allowance for U.S. Appl. No. 12,798,995, dated May 9, cation is Coming to Thousands of Mobile Phone Retail Stores
2012 . Nationwide” , fastmobile Press Release , Sep. 15 , 2003, pp . 1-3 .
Notice of Allowance for U.S. Appl. No. 12 /798,995, dated Oct. 3 , IMBOT, Press Release , “ IMBOT offers new Text 2 Voice Service
2012 .
Text 2 Voice service enables wireless customers to send voice
messages from 2 -Way devices ” , Oct. 29 , 2001 , pp . 1-2 .
Notice of Allowance for U.S. Appl. No. 12/798,995 , dated Jul. 16 , Internet Traveler, “ Welcome to the Inter.Net Communicator Tour !" ,
2013 . http://www.inter.net/traveler/tour/communicator_messaging.php , down
Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Oct. 30 , loaded Oct. 14 , 2003, p . 1.
2013 . J. Rosenberg , H. Schulzrinne, Internet Draft, “ SIP for Presence,"
Notice of Allowance for U.S. Appl. No. 12 /798,995 , dated Feb. 20 , http://www.alternic.org/drafts/drafts-r-s/draft-rosenberg=sip-pip-00 .
2014 . txt, Nov. 13, 1998 , Bell Laboratories, Columbia , pp . 1-31 .
Office Action for U.S. Appl. No. 14 /272,632 , dated Jul. 27, 2015 . Joseph , Anthony D. et al., “ The Case for Services over Cascaded
Notice of Allowance for U.S. Appl. No. 14 /272,632 , dated Sep. 18 , Networks” , EECS Department, CS Division , University of Califor
2015 . nia , Berkeley , http://iceberg.cs.berkeley.edu/, International Confer
Office Action for U.S. Appl. No. 14 /922,344 , dated Apr. 27 , 2016 . ence on Wireless and Mobile Multimedia 1998 , pp. 1-9 .
Office Action for U.S. Appl. No. 14 /922,344 , dated Oct. 7, 2016 . MobileShop , “SMS also know as text messaging” , http ://www .
Notice of Allowance for U.S. Appl. No. 14 /922,344 , dated Feb. 14 , mobileshop.org/howitworks.sms.htm , downloaded Oct. 14 , 2003 ,
2017 . pp . 1-2 .
Notice of Allowance for U.S. Appl. No. 14 /922,344 , dated Mar. 6 , Schulzrinne , H. et al., RPID : Rich Presence Extensions to the
2017 . Presence Information Data Format (PIDF), draft - ietf= simple -rpid
Notice of Allowance for U.S. Appl. No. 15/ 469,440 , dated Sep. 6 , 06.txt, Jun. 2 , 2005, http://www1.ietf.org/mail-archive/web/simple/
2017 . current/msg05398.html, downloaded Nov. 15 , 2006 , pp . 1-35 .
First Office Action for CN Patent Application No. 200680027964.9 , Schulzrinne, H. et al., RPID : Rich Presence Extensions to the
dated Mar. 26 , 2010 ( 17 pages ). Presence Information Data Format (PIDF), draft -ietf-simple -rpid
10.txt, Dec. 20 , 2005 , pp . 1-41 .
Second Office Action for CN Patent Application No. 200680027964 . Schulzrinne , H. et al., RPID : Rich Presence Extensions to the
9 , dated Oct. 25 , 2010 (14 pages ). Presence Information Data Format (PIDF), draft-ietf-simple -rpid
Third Office Action for CN Patent Application No. 200680027964. 10.txt, Dec. 4 , 2005 , pp . 1-35 .
9 , dated Apr. 8, 2011 ( 11 pages). Sonim Technologies, Inc., “ Integrated voice and text messanging
Notice ofRejection for CN Patent Application No. 200680027964. over GPRS showcased jointly by Sonim , Symbian and Texas
9 , dated Jan. 6 , 2012 ( 11 pgs.). Instruments” , Sonim Press Release , Dec. 2 , 2002, pp . 1-2 .
“ Company Overview ”, http://www.fastmobile.com/company_overview . Symbian Ltd., “ Symbian OS Version 7.0 : Functional description ” ,
html, downloaded Nov. 5 , 2003, p . 1. Revision 1.5 , Feb. 2003 , pp . 1-24 .
“ Introducing the Tellme Voice Application Network ” , Tellme, http : // Symbian Ltd., “ Symbian OS Version 7.0s: Functional description " ,
www.tellme.com/products/, downloaded Oct. 2 , 2003 , p . 1. Revision 2.1 , Jun . 2003 , pp . 1-29 .
" Iotum History,” Iotum Corp., http://iotum.com/simplyrelevant/ Symbian Ltd., “ Technology : Creating Symbian OS phones” , http ://
2006 /04 /03/iotum -history/, downloaded May 15 , 2006 , pp . 1-4 . www.symbian.com/technology/create-symb-OS-phones.html, down
"Messaging ", Vodafone Group , 2001, http:www.vodafone.co.nz/ loaded Nov. 5 , 2003, p . 1-8 .
business/ 10.2.3_messaging.jsp , downloaded Oct. 14 , 2003 , pp . 1-2 . Symbian Ltd., “ Technology : Why is a different operating system
“Microsoft Windows Messenger: Go Beyond Text with Voice & needed ”, http://www.symbian.com/technology/why-diff-os.html,down
Video Chats” , Dell Inc., http://www.dell.com/us/en/dhs/topics/ loaded Nov. 5 , 2003, pp. 1-5 .
segtopic_002_xp_im.htm , downloaded Oct. 2 , 2003, pp . 1-2 . Verizon Wireless,“ TXT messaging” , http://www.vtext.com/customer
“Microsoft WindowsMessenger: Instantly Communicate with Fam site/jsp /messaging_lo.jsp , downloaded Oct. 2 , 2003 , p . 1.
ily and Friends Messenger” , Dell Inc., http://www.dell.com/us/en/ W3C , Voice Extensible Markup Language (VoiceXML ) Version
2.0 , W3C , www.w3.org , Feb. 20 , 2003 .
dhs/topics/segtopic_001_xp_im.htm , downloaded Oct. 2, 2003, pp . Yahoo!Messenger, “ Yahoo!Messenger Talk for Free !" , http ://
1-3. messenger.yahoo.com/messenger/help/voicechat.html, downloaded
“ Our Solution ,” Iotum Corp., http://www.iotum.com/our_solution . Oct. 2 , 2003 , pp . 1-2 .
php , downloaded May 15 , 2006 , pp . 1-2 . Notice of Allowance for U.S. Patent Application No. 15/469,440 ,
Short Message Service /Interactive Voice Response (SMS/ IVR ), dated Dec. 19 , 2017 .
Lucent Technologies, 2003 , pp . 1-2 . Notice of Allowance for U.S. Appl. No. 15 /469,440, dated Sep. 27,
“ Text messaging” , Vodafone Group , 2001, Vodafone Services, 2018 .
" All abouttextmessaging ”, http://www.vodafone.co.nz/services/07. Notice of Allowance for U.S. Appl. No. 15 /469,440 , dated Aug. 8 ,
a.1_two_way_messaging.jsp ? hd = 4yourbusiness & ..., downloaded 2018.
Oct. 14 , 2003, pp . 1-2 . Bulk , F. “ Final Project: Skype," http://www1.cs.columbia.edu/
“ We bring relevance to communications,” Cnet News, Ina Fried , ~ salman / skype/ frank.pdf, May 5 , 2004, pp . 23 .
Jul. 21, 2005, pp . 1-2 . Using AIM on Windows Video IM , https://web.archive.org/web/
Appenzeller, et al., “ The Mobile People Architecture” , Technical 20050307231707/http :/www.aim.com/help_faq/using/win/video_im .
Report: CSL - TR -00000 , Computer Systems Laboratory, Depart adp ? aolp = , downloaded Aug. 9 , 2019 , 2 pgs.
ments of Electrical Engineering and Computer Science, Stanford Download AIM for Windows, https://web.archive.org/web/
University, Jan. 1999 , pp . 1-13 . 20050204020358 /http :/channels.netscape.com/wrap/linker.jsp ? floc =
US 10,492,038 B2
Page 4
( 56 ) References Cited MSN Messenger, “Most Frequently Asked Questions,” https: //web .
archive.org/web/20050601014205/http://messenger.msn.com/
OTHER PUBLICATIONS Help /, downloaded Aug. 9 , 2019 , 8 pgs .
Skype TM , “ Skype is free Internet telephony that just works,”
at_oswin_1_12 & ref= http ://www.aim.com/get_aim/win/latest_win . https://web.archive.org/web/20050601003206/http://www.skype.
adp ? aolp = # whatsnew , downloaded Aug. 9, 2019 , 2 pgs . com /, downloaded Aug. 9 , 2019 , 21 pgs .
Using AIM on Windows - Mar. 8, 2005 , https://web.archive.org/ Skype TM , “ How to Use Skype," https://web.archive.org/web/
web /20050308012857/http : //www.aim.com/help_faq/using/win/ 20041229163311/http:/www.skype.com/help/guides/usingsky pe.html,
aimtalk.adp ?aolp = , downloaded Aug. 9, 2019, 2 pgs. downloaded Aug. 9 , 2019 , 6 pgs.
AIM Buddy List , https://web.archive.org/web/20050531083304/ Skype TM , “ How to Add a Contact User Guide ,” https://web .
archive.org/web/20041230160101/http:/www.skype.com/help/guides/
http://www.aim.com/help_faq/starting out/buddylist.adp ? aolp = , down adduser.html, 10 pgs.
loaded Aug. 9, 2019 , 1 pg. Skype TM , “Registering a Skype Name User Guide,” https: //
AIM Registration , https://web.archive.org/web/20041216085601/ web.archive.org/web/20041229042631/http:/www.skype.com/help/
http :/wvvw.aim.com :80 /help_faq/starting_out/registration.adp ? guides/registration.html, 10 pgs .
aolp = , downloaded Aug. 9, 2019 , 2 pgs. “ Yahoo!Messenger,” https://web.archive.org/web/20050601042101/
Using AIM on Windows Mar. 5 , 2005, https://web.archive.org/ http://messenger.yahoo.com/newtoim.php , downloaded Aug. 9 , 2019 ,
web /20050305095328 /http : /www.aim.com :80 /help_faq/using /win / 3 pgs.
instant_message.adp ?aolp = , downloaded Aug. 9, 2019, 2 pgs . “ Yahoo! Help — All -New Messenger 6.0 ," https://web.archive.org/
AIM® Inside the Sidekick , https://web.archive.org/web/ web/20040806142252/http://help.yahoo.com/help/us/messenger/w
20050519114310 /http :/mymobile.aol.com/portal/im/pdfs/tmobile/ intabuse /abuse -02.html, downloaded Aug. 9 , 2019 , 2 pgs .
AIM_SIDEKICK_UG_tmobile.pdf, copyright 2002 by America “ AOL 8 Instant Messenger TM ,” https://web.archive.org/web/
Online, Inc., 13 pgs. 20050601001345/http :/www.aim.com/, downloaded Aug. 6 , 2019 ,
2 pgs.
MSN Messenger:mac, https://web.archive.org/web/20050604080622/ MSN Messenger Version 7.0 ,“Messenger," https://web.archive.org/
http://www.microsoft.com/mac/default.aspx?pid=msnmessenger ,down web / 20050601023444 /http :/messenger.msn.com downloaded
loaded Aug. 9 , 2019 , 2 pgs. Aug. 6 , 2019 , 2 pgs.
MSN Messenger, “ Communicate with MSN® Messenger,” https:// Skype TM , “ Skype is free Internet telephony that just works,”
web.archive.org/web/20051013055708/http://www.imagine-msn . https://web.archive.org/web/20050601003206/http:/www.skype.
com /messenger/post/ communicate/ instantmessage.aspx , down com , downloaded Aug. 6 , 2019 , 3 pgs .
loaded Aug. 9, 2019 , 2 pgs. " Yahoo!Messenger," https://web.archive.org/web/20050601012258/
MSN Messenger, “ Download MSN Messenger," https: //web.archive . http :/messenger.yahoo.com , downloaded Aug. 6 , 2019 , 1 pg.
org/web /20050601002632/http://messenger.msn.com/download/, down
loaded Aug. 9 , 2019 , 1 pg . * cited by examiner
U.S. Patent Nov. 26 , 2019 Sheet 1 of 9 US 10,492,038 B2
ICM Default
***
Mobile phone Voice mail
2 Office phone Voicemail
3 Home phone Voice mail
5 Home/office SMS (to office /home PC ) Email
6 Mobile Online chal (10 mobile phone or PDA ) Voice mail
HomeOnline chat (NetMeeting, AOL , ICQ etc.) Voice mail
Voice mail with instantnotification to mobile devices of the user
Office fax
Home fax Reject
Mobile Email (Blackberry etc. ) Email
Email Reject
4 User defined

ContactClass2 Relatives and friends
ContactClass3 Boss and VIP
ContactClass Secretary
ContactClass10 User defined
UrgClass1 Life threatening - interrupt at any time and occasion

UrgClass2 Urgent confirmed meeting reminder - interruption
allowed
UrgClass 3 Urgent matter requiring immediate attention
UrgClass4 Important matter requiring quick attention
UrgClass 5 Regular work related matter
UrgClass Casual contact
UrgClass 7 Cold calls from unknown person
UrgClass User defined
FIGURE 3
MyBusy Statel Important meeting

MyBusyState2 Ordinary meeting
MyBusy State4 Sleeping
MyBusy State5 Resting
MyBusy State User defined
ContactClass UrgClass MyBusy State ICM allowed

ContactClass2 UrgClass 1-3
UrgClass4-6 MyBusyStatel
MyBusyState2-3
MyBusyState4-5
UrgClass 7-8 All ICM 13
100
106
102
NETWORK
108
104
104
106 102
FIG . 6
START 200
202
INCOMING
VOICE
NO
CALL
?
YES
204
206 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE
212
CALL
AUDIO
MESSAGE 214
? YES
ENGAGE IN
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 216
NO ?
TEXT
MESSAGE 218
YES ? YES 2
OBTAIN AND SEND
TEXTMESSAGE
NO TO CALLER
220
VOICE 222
MAIL YES
? DIRECT TO
NO VOICE MAIL
DROP THE INCOMING

VOICE CALL 224
FIG . 7
END
300
START
PRESENT ONE OR MORE PREDETERMINED

304
AUDIO MESSAGES
306
PREDET .
AUDIO
MESSAGE NO
SELECTED 310
?
CUSTOM
YES
AUDIO
308 MESSAGE
REQUESTED NO
?
PLAY SELECTED AUDIO
MESSAGE FOR CALLER
YES
RECORD CUSTOM AUDIO 312

MESSAGE
PLAY CUSTOM AUDIO

MESSAGE FOR CALLER 314
END
FIG . 8
400
START
DROP THE INCOMING VOICE CALL 402
DISPLAY ONE OR MORE PREDETERMINED

TEXT MESSAGES 404
406
PREDET .
TEXT
MESSAGE NO
SELECTED 410
?
YES
CUSTOM
TEXT
408
MESSAGE
REQUESTED NO
TRANSMIT SELECTED ?
TEXT MESSAGE
TO CALLER
YES
ENTER CUSTOM
TEXT MESSAGE 412
TRANSMIT CUSTOM
TEXT MESSAGE 414
TO CALLER
END
FIG . 9
START
500
202
INCOMING
VOICE
CALL NO
?
YES
204
206 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE
CALL 502
HEAD - SET 214
ACTIVE YES
ENGAGE IN ?
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 504
NO ?
DISPLAY 218
YES PRESENT YES
? OBTAIN AND SEND
TEXT MESSAGE
NO TO CALLER
220
VOICE 222
MAIL YES
? DIRECT TO
NO VOICE MAIL
DROP THE INCOMING
224
FIG . 10 VOICE CALL
END
600
START
602
INCOMING
TEXT
MESSAGE NO
?
YES
604
AUDIO
OR TEXT
PRESENTATION 610
TEXT
?
TEXT MESSAGE
AUDIO
606 CONVERT THE INCOMING TEXT

END
FIG . 11
START
700
702
REPLY
MESSAGE
NO TO BE
SENT 714
?
YES DISPLAY TEXT
ENTRY SCREEN
704
AUDIO
OR TEXT
MESSAGE TEXT
TO BE MESSAGE
TEXT ENTERED NO
FORMED
? ?
706 716 YES
AUDIO
708
AUDIO
MESSAGE
RECORDED NO
?
710
YES

TO A TEXTMESSAGE
TRANSMIT THE TEXT MESSAGE

OVER A WIRELESS NETWORK 712
END FIG . 12
US 10,492,038 B2
1 2
METHOD AND APPARATUS TO MANAGE It should be apparent from the foregoing that there is still
MESSAGING PROVIDING DIFFERENT a need to help manage the numerous modes of communi
COMMUNICATION MODES DEPENDING ON cation .
ONE IDENTIFIER AND NOT REQUIRING TO SUMMARY OF THE INVENTION
DISCLOSE CONTACT INFORMATION 5
CROSS -REFERENCE TO RELATED Different embodiments of a computer- implemented sys

APPLICATIONS tem and method to manage the communication of a user are
disclosed . In one embodiment, an apparatus , using at least a
This application is a continuation of U.S. patent applica network
10
vide a
-based portal based on Internet protocol, could pro
number of communication options to a first user, with
tion Ser. No. 15 /469,440 , filed Mar. 24 , 2017 , and entitled
“ A NETWORK -BASED PORTAL TO MANAGE COM messages eligible to be received via an electronic device
MUNICATION ,” , which is hereby incorporated herein by associated with a second user based on any of the options
depending on an identifier associated with the second user,
reference ,which application is a continuation of U.S. patent 15 the options including text messaging and voice communi
application Ser. No. 14 /922,344 , filed Oct. 26 , 2015 , now cation; could receive an indication from the first user via an
U.S. Pat. No. 9,736,664, and entitled " SYSTEMS AND electronic device associated with the first user, indicating the
PROCESSES TO MANAGE MULTIPLE MODES OF selection of a communication option for a message for the
COMMUNICATION ,” which is hereby incorporated herein second user; could permit the second user to block the first
by reference, which application is a continuation of U.S. 20 user from accessing the second user; and could determine
patent application Ser. No. 14 /272,632, filed May 8 , 2014 , availability of the second user to receive themessage . In the
now U.S. Pat. No. 9,204,268, and entitled “ SYSTEMS AND embodiment, the apparatus could require contact informa
PROCESSES TO MANAGE MULTIPLE MODES OF tion associated with the second user to allow the second user
COMMUNICATION ,” which is hereby incorporated herein to receive messages via the network -based portal , with the
by reference , which application is a continuation of U.S. 25 contact information associated with the second user not
patent application Ser. No. 12 /798,995 , filed Apr. 14 , 2010 , provided to the first user via the electronic device associated
now U.S. Pat. No. 8,744,407, and entitled “ SYSTEMS AND with the first user , even when the message is received by the
PROCESSES TO MANAGE MULTIPLE MODES OF second user via the electronic device associated with the
COMMUNICATION ,” which is hereby incorporated herein second user, and with the contact information associated
by reference, which application is a continuation of U.S. 30 with the second user being distinct from the identifier
patent application Ser . No. 11/452,115 , filed Jun . 12, 2006 , associated with the second user .
now U.S. Pat. No. 7,729,688 , and entitled “ SYSTEMS AND userA . personIn one
tries to electronically convey a message to the
embodi ent, the status of the user is identified ;
PROCESSES TO MANAGE MULTIPLE MODES OF the identity of the person
COMMUNICATION ” , which is hereby incorporated herein 35 message is identified ; the isaccess identified ; the urgency of the
priority of the person is
by reference, which application is a continuation - in -part determined based on the person's identity ; and a process is
application of U.S. patent application Ser. No. 11/006,343 , set to manage the message using one or more rules , and in
filed Dec. 7 , 2004, now U.S. Pat. No. 7,116,976 , and entitled view of the status of the user, the access priority of the
“ ADAPTABLE COMMUNICATION TECHNIQUES FOR person and the urgency of the message .
ELECTRONIC DEVICES,” which is hereby incorporated 40 Based on different embodiments, the status of the user
herein by reference , which claims priority to U.S. Provi depends on the current activity or location of the user , or the
sional Patent Application No. 60 /527,565, filed Dec. 8, current time. The status of the user can also be defined by the
2003 , entitled “ ADAPTABLE COMMUNICATION TECH user. Similarly, the access priority of the person can be
NIQUES FOR ELECTRONIC DEVICES,” and which is defined by the user, or is set depending on the user's reaction
hereby incorporated herein by reference . 45 towards a prior message from the person. Also , the urgency
This application, by way of U.S. patent application Ser . of the message is set by the person .
No. 11 /452,115 , also claims priority to U.S. Provisional The process can depend on the mode of communication of
Patent Application No. 60 /689,686 , filed Jun . 10 , 2005 , the message . For example , the mode of communication can
entitled “ SYSTEMS AND PROCESSES TO MANAGE include a mobile phone, an office phone, a home phone , a
MULTIPLE MODES OF COMMUNICATION,” andwhich 50 home
mobile/office
SMSSMS
, a pager from a mobile phone or PDA , a
, mobile online chat, home online chat, a
is hereby incorporated herein by reference . voice mail with /without instant notification , an office fax , a
BACKGROUND OF THE INVENTION home fax , a mobile email, and an email.
In one embodiment, the user receives the message
For many years, other than mails from post offices, we 55 through
another
a handheld device, such as a cellular phone. In
embodiment , the message is electronically conveyed
typically only received information from afar through tele based on Internet protocol through a website .
phones. However, in the past few years ,ways that others can
send us information have increased significantly. Just to list receive the message , the persontheisprocess
In one embodiment , though allows the user to
not aware of the contact
a few differentmodes of communication ,we can be reached information of the user. For example, the person is not aware
from standard desk phones, fax , cell phones, electronic 60 of the phone number of the cellular phone that the user used
mails, and instantmessages. In addition , we can have more to talk to the person . This prevents the person from directly
than one phone number and multiple electronic mail accessing the user without going through an intermediate
addresses. There are people we like to communicate with , control, such as a website . Similarly, the user does not have
and there are those we prefer to avoid . Managing informa- 65 to be aware of the contact information of the person .
tion from all such different modes can be quite time con In another embodiment, the defined access priority of the
suming person is stored at a website , allowing the website to access
US 10,492,038 B2
3 4
such information without asking for the user's permission . In one embodiment, the user can get appropriate notifi
In one embodiment, the defined access priority is stored in cation on the source of the incoming communication
a private database under the user's control. request. The attributes of the notification can depend on the
In one embodiment, textmessages could be received in an urgency of the communication and/or the status of the user.
audio manner, and audio messages could be sent as text 5 The user may receive information from differentmodes of
messages. communication. For example , the user can have mobile
Other aspects and advantages of the present invention will phones , fixed lines at home or office , emails, SMS, and
become apparent from the following detailed description , faxes , with their different numbers and/or addresses . One
which , when taken in conjunction with the accompanying embodiment can help the user efficiently manage informa
drawings, illustrates by way of example the principles of the 10 ber
tionone
fromspecific
the different
addressmodes
from .one
Themode
user only has to remem.
of communication
invention .
Through that address, the user can receive communications
BRIEF DESCRIPTION OF THE DRAWINGS from allmodes of communication , independent of where the
user is, or the type of hardware the user has. This allows the
FIG . 1 shows a number of intelligent communication 15 numerous
user to efficiently maintain his communication from the
modes even when he is traveling. For example ,
modes according to one embodiment of the invention .
FIG . 2 shows a number of contact classes according to numbers ) whennot
the user does have to change phones (and the phone
he moves from areas covering 3G to areas
one embodiment of the invention . that do not.
FIG . 3 shows a number of urgency classes according to 20 A number of embodiments depend on the differentmodes
FIG . 4 shows a number of statuses of a user according toplatform . A communication gateway or a portal is formed
FIG . 5 shows one embodiment of an example of an sources through differentmodes . This , in turn , could reduce
Access Priority Database according to one embodiment of 25 the numerous addresses the user has to remember, to one
the invention . address. For example, an e -mail address for the user can
embodiment of the invention . addresses from different communication modes . The access
FIG . 7 is a flow diagram of a personal call response identifier can become the user's digital identity. In one
process according to one embodiment of the invention. 30 embodiment, the user's other types of identification , such as
FIG . 8 is a flow diagram of an audio message response the user's driver licenser number, can be the user's access
process according to one embodiment of the invention . identifier.
process according to one embodiment of the invention. based on the web . Based on the portal, the user can securely
FIG . 10 is a flow diagram of an automated call response 35 determine who can reach him at what conditions. This can
process according to one embodiment of the invention . be done based on a status indicator. As an example, this
process according to one embodiment of the invention . the access priorities of the person trying to reach the user (or
FIG . 12 is a flow diagram of a message presentation the relationship or the lack of relationship between the user
process according to one embodiment of the invention. 40 and the person ), and/or the urgency of the message from the
Same numerals in FIGS. 1-12 are assigned to similar person . The status of the user can be dynamically deter
elements in all the figures. Embodiments of the invention are mined , based on the current condition (s ) of the user. The
discussed below with reference to FIGS. 1-12 . However , portal can allow the user and the person to select different
those skilled in the art will readily appreciate that the options, which can be modified as desired . For example , the
detailed description given herein with respect to these fig- 45 relationship can be preset by the user and stored in a
ures is for explanatory purposes as the invention extends database , while the urgency of themessage can be set by the
beyond these limited embodiments . person .
Thus , in one embodiment, the portal can be used to
DETAILED DESCRIPTION OF THE control the selection and setting of different intelligent
INVENTION 50 communication modes for the user . These intelligent com
munication modes allow priorities of various kinds of com
One embodiment of the invention can automatically munication options to be set by the user. The portal allows
remove unwanted communications. Certain communica worldwide access to the user , and can dynamically deter
tions are relatively easy to determine to be unwanted , such mine, for example, whether a call initiated at different time
as marketing cold calls and wrong number calls . Other 55 by different callers should be accepted by the user in
communications may be more difficult. They can depend not real-time or handled by other mechanisms. From this infor
conditions or status of the receiver (a user ) of the commu example , into different degrees of undesirability . Some
nication . The status can be related to the user's current requests can be automatically blocked from the user. Others
activity and /or location . For example, when the user is on a 60 can be diverted and handled by other mechanism , such as
train going to work , the user probably does not mind diverting a phone call to an email or voice mail.
chatting with his grandchild . However, if the user is having In one embodiment, the portal or gateway also includes a
his yearly review meeting with his boss , the user probably database to keep track of the user's different contacts or
would prefer to avoid the call from his grandchild , unless it acquaintances , and the access priorities of each contact. The
is an emergency. Based on the embodiment, communica- 65 user can modify information in the database , such as assign
tions from sources the user wants to postpone receiving can ing and /or changing the priorities of the contacts . Based on
be automatically diverted . the information (or lack of information ) in the database of
US 10,492,038 B2
5 6
the contact trying to access the user, and based on the status
the identity of the caller, and the status and schedule of the
of the user, the gateway can automatically select an intelli
user. For example, the appointment can be for a conference
gentmode of communication for the user. This selection can call.
be done dynamically . To illustrate , in one embodiment, a portal provides a
In one embodiment, the portal can dynamically change 5 number of intelligent communication modes (ICM ) for the
the access priorities of a caller trying to reach the user. For user to select as shown in FIG . 1. There are three columns
example , previously the caller is of high priority to the user, in the table . If the communication mode selected in the
and the user has set her access priorities accordingly . Lately, second column does not work , the portal automatically
every time the caller trying to reach the user , the request was 10
defaults to the corresponding approach in the third column .
denied . After a preset number of rejections, the portal can For example , under ICM 1, if the mobile phone is busy,
automatically send a message to the user, asking the user if default to voice mail. Someof the selections do nothave any
the user would like to lower the access priority of the caller. default because it may not be necessary to default. For
If the response is affirmative , the caller's priority is auto example , under ICM 8, the incoming message goes directly
matically reduced . 15
to voice mail with instant notification to mobile devices of
In another embodiment, the user does not have to set the user. The incoming message can usually go to voice
priorities of each contact. The system monitors every call, mail. There is no need to default.
and provides the contact's identity to the user. Based on the As a receiver of communication , the user can define a
user's reaction to the call (e.g. accepting or rejecting it ), the number of contact classes, as shown in FIG . 2. The user can
system automatically sets the contact's priorities. In one 20 user set up a number of urgency classes, as shown in FIG . 3. The
embodiment, the system can then query the user for approval basedcanondefine a number of status, as shown in FIG . 4. Then ,
on the setting, and allow the user to adjust it as necessary. In Priority Databasein for
tables FIGS . 1-4 , the user can set up an Access
different ContactClasses, as shown in
another embodiment, the system can continue to modify the FIG . 5 .
caller's priorities based on the user's reaction to the caller's As another example , the user can categorize the following
subsequent calls. 25
contacts into the corresponding ContactClasses:
believes to be sensitive local in a different database . Such
information can be stored securely under the user's direct Alice (Wife)
Peter ( Close Friend)
ContactClass1
ContactClass2
control. The portal can retrieve information from the differ 30 Colina (Close Friend's ContactClass2
ent database when required . In another embodiment, the user wife )
can restrict or limit such retrieval process.
Additional confidentiality can be provided . In one Peterwants to make a mobile phone call the user. In one
embodiment, using phone calls as an example , the user can embodiment , Peter calls a portal. As an example , the portal
be aware of the identity of the caller even without being 35 can be the user's
informed of the number of the caller. Similarly , the caller can identity to be PeterISP .
. The portal first verifies the caller's
This can be done, for example, by a
reach the user without being aware of the number of the public key challenge based on Peter having a public key
phone the user is using to receive the call . The user can keep digital certificate . In another example , Peter is also a regis
his location and/ or status confidential but still can receive tered user of the portal. Then , Peter's identity can be more
the communication . This can be useful because there are 40 readily identified or verified .
situations, for example, when the user does not want to In one embodiment, after verification , a virtual address /
disclose his contact information but the user needs to receive number for the communication session is created allowing
services provided by the caller. Peter to reach the user, which can be by phone . The user's
One approach to maintain such confidentiality while phone number does not have to be disclosed to Peter.
maintaining real-time communication is based on a system 45 Similarly, Peter's mobile phone number does not have to be
that digitally identifies the identities of the caller and the disclosed to the user. The portal can assure the user that the
receiver. Note that the term caller is used in general. It is not person calling is Peter based on an identification verification
just limited to phone calls, but they can be any person or process , such as ones described above .
entity requesting to communicate with the user , such as In establishing contact, the portal can access the user's
trying to send a message to the user. As a separate note , the 50 database and determine thatPeter belongs to ContactClass2.
caller can also be a user of different embodiments of the The database can , for example , be in the portal.
invention . In another embodiment, the database is in a personal
After determining the identities , the system can establish communication device of the user . The portal accesses the
connections between the caller and the user in real time. personal communication device to determine Peter's Con
Though contacts are established , the system only needs to 55 tactClass.
ensure the identities of the caller and the user to each other. Based on the ContactClass information , the status of the
However , the system does not have to disclose the phone user and Peter's urgency setting , the user may receive
numbers , electronic addresses , physical locations and /or Peter's call directly. As another example, Petermay be asked
other attributes of the caller and the user to each other. In one to leave a voice mail to the user, while the user is notified by
embodiment, real time implies that the time required for the 60 a mobile short message regarding an incoming call from
identification is similar to the typical timerequired to set up , Peter .
for example , a telephone call. The system can be a portal As additional examples , in one embodiment, location
based on the web . information of the user could be determined based on GPS
In one embodiment, a portal also holds the user's elec information from , for example , the user's cell phone .
tronic calendar. The calendar can be programmable, with 65 In one embodiment, the user receives messages through a
entries set by the user. The portal can automatically and handheld device , such as a phone, and the phone has a
securely set appointments for the user since the portalknows switch . The switch can be a physical button or a software
US 10,492,038 B2
7 8
setting, such as a pull-down menu. The user could set his Further, the computer - implemented methods and systems
status dynamically by changing the physical or logical discussed above can be used in conjunction with one or more
position of the switch . For example, one position can of the various approaches discussed in U.S. patent applica
indicate that the user is very busy, and should only be tion Ser. No. 11/006,343 . For example, the automated
interrupted by an urgent message from the user's closest 5 actions or decisions (e.g., intelligent secretary, decision 204
contacts , such as his wife or parents . Another position can in FIG . 2 , etc.) of U.S. patent application Ser. No. 11/006 ,
indicate that the user's status allows the user to receive any 343 can be automatically made by the systems/methods
messages from anyone . described above . Still further, the various approaches dis
As explained above, based on an embodiment, a message cussed in U.S. patent application Ser. No. 11/006,343 can be
is electronically conveyed by a central network server , such 10 used in conjunction with one or more the various methods/
as a web server based on Internet protocol. A portal or systems discussed above. For example , the systems/methods
gateway approach could provide general Internet access to described above can use the messaging approaches (e.g.,
one or more embodiments of the communication manage audio or textual messages) described in U.S. patent appli
ment systems so that users can configure the system behav cation Ser. No. 11 /006,343 .
ior they desire . The portal or gateway can then facilitate 15 Different embodiments of the invention pertain to
download of a database or update thereto to a communica improved approaches for users of electronic devices to
tion device, such as a phone. communicate with one another . The electronic devices have
Also , as explained above, based on an embodiment, a user audio and/or textual output capabilities. The improved
could efficiently maintain his communication, and does not approaches can enable users to communicate in different
even have to change phones when he moves from areas 20 ways depending on device configuration , user preferences ,
covering 3G to areas that do not. These phones could be prior history, time or other criteria . In one embodiment, the
based on different communication mechanisms, such as communication between users is achieved by short audio or
GSM , CDMA, 3G and 4G systems. Also as explained above , textualmessages.
the user could keep information in local databases, such as The electronic device can be any computing device hav
in such a phone . For example , the intelligent communication 25 ing communication capabilities . Such computing devices
modes shown in FIG . 1 for the user to select are in the phone . can be referred to as communication devices. Examples of
The user could define the contact classes , such as the ones electronic devices include personal computers, personal
shown in FIG . 2; set up the urgency classes, such as the ones digital assistants , pagers or mobile telephones.
shown in FIG . 3 ; define the statuses , such as the ones shown Embodiments of the invention are discussed below with
in FIG . 4 ; set up the Access Priority Database , such as the 30 reference to FIGS. 6-12 . However , those skilled in the art
one shown in FIG . 5 ; and categorize a number of the user's will readily appreciate that the detailed description given
contacts into the corresponding ContactClasses, all in the herein with respect to these figures is for explanatory
phone . When a caller places a call the phone, based on purposes as the invention extends beyond these limited
information previously set in the phone and based on the embodiments .
urgency class selected by the caller , the phone could auto- 35 FIG . 6 is a communication system 100 according to one
matically manage the communication . Note that the phone embodiment of the invention . The communication system
does nothave to be a cellular phone . In one embodiment, the 100 can support different communication devices , including
phone is a desk top phone. mobile telephones 102 , computers 104 (e.g., personal com
Again as explained above , the person or the caller trying puters ) and/ or wireless personal digital assistants (PDAs)
to contact the user could select different options. For 40 106. Users of the communication devices 102-106 can
example, the urgency of themessage can be set by the caller. communicate with like or different communication devices .
This selection is typically in the call setup phase . In one Each communication device 102-106 offers one or both of
embodiment, the caller has pre -selected the urgency class audio or textual communication capabilities. These commu
before making the call . In another embodiment, if the caller nication devices 102-106 can inter-communicate with one
has not selected the urgency class , the system could prompt 45 another through a network 108. The network 108 can
the caller to input an urgency class or status before the call include one or more of voice networks and data networks.
or message is routed to the user. In yet another embodiment, For example , one network is a data network providing a
different urgency classes could be defined by the caller. slow speed data channel for transmission of Short Message
Different embodiments or implementations may yield Service (SMS) messages (which are typically limited to 160
different advantages . One advantage is that communications 50 text characters) to a Short Message Service Center (SMSC )
for users of electronic devices can be flexibly provided . and then forwarded on to the destination . Besides short
Another advantage is that communication mode changes can messages (e.g., SMS messages ), the network 108 can also
be performed at an electronic device to better suit the needs support other messaging protocols for sending and receiving
or condition of the electronic device or user preferences. In enhanced messages ( EMS), multimedia messages (MMS),
still another advantage , a user can provide feedback to a 55 email and fax messages. Other networks support faster data
caller without answering a voice call from the caller. channels and voice channels , such as GPRS , UMTS , G4,
Another aspect of the invention pertains to improved GSM , CDMA and various protocols, such as UDP, TCP,
approaches to respond to an incoming text message . The WAP, PDP other protocols .
improved approaches enable a recipient to provide a reply According to one embodiment of the invention , one of the
message to an initiator. The incoming text message can be 60 communication devices 102-106 can send a shortmessage to
presented to the recipient with an audio or textual presen another of the communication devices 102-106 . The short
tation . Thereafter, a reply text message can be sent back to message can be text-based or audio -based . The sending
the initiator. The recipient can form the reply text message communication device allows its user to create the short
by recording a brief audio message or entering a text message as the user desires and /or as the device permits . For
message . In the case in which a brief audio message is used , 65 example , the user might interact with a keypad or keyboard
the audio message can be automatically converted to a text to enter the short message, or the user might record audio
message before being transmitted to the initiator. inputs (e.g., speech ) for the short message . The short mes
US 10,492,038 B2
9 10
sage can then be sent to the receiving communication voice mail . On the other hand , when the decision 220
device. The sending of the short message may involve determines that the incoming voice call is not to be directed
converting the short message from an audio message to a to voice mail, the incoming voice call is dropped 224 .
text message, or vice versa . Also , the receiving communi Following the blocks 214 , 218 , 222 and 224 , the personal
cation device can further convert the short message from 5 call response process 200 is complete and ends.
audio -to -text or from text-to -audio . In any case , the short In another embodiment, a personal call response process
message is presented (e.g., displayed or played) to the user could announce the calling party to the called party (user).
of the receiving communication device . The presentation In announcing the calling party , the personal call response
can vary as the user desires or as the device permits. process would present the called party with information
One aspect of the invention pertains to improved 10 pertaining to the calling party ( e.g., display or audio sound ).
approaches to respond to incoming voice calls. The Such information could , for example , help the called party
improved approaches enable a called party (i.e., a party to decide whether to answer the incoming voice call. The
being called ) to provide some information to a calling party information can , for example, include one or more of name
without directly engaging in a voice call with the calling (individual or business ), telephone number, or other caller
party . The called party can choose not to take the voice call 15 identification . The information could also include status
from the calling party . Instead , the called party can provide information of the calling party, such as position , health ,
the calling party with some limited information . The limited mood , etc. As an example , the information could be pre
information can be provided in an audio or textual format. sented to the user prior to the decision 204 of the personal
In one embodiment, the limited information provides the call response process 200 shown in FIG . 7 .
calling party with feedback as to why the voice call was not 20 In still another embodiment, an automated decision pro
taken . cess to decide whether to answer a call can be based on time
FIG . 7 is a flow diagram of a personal call response (e.g., decision 204 ). For example , the called party can
process 200 according to one embodiment of the invention . previously set a rule , such as that from midnight to 6 am , the
The personal call response process 200 is performed by an party does not want to answer voice calls. Then , during this
electronic device, such as a mobile communication device 25 time period , the electronic device can automatically decide
(e.g., mobile telephone ). The personal call response process not to answer incoming calls . In one implementation , when
200 begins with a decision 202 that determines whether the electronic device decides not to answer incoming calls,
there is an incoming voice call. When the decision 202 no indication of incoming calls will be provided to the called
determines that there is no incoming voice call , then the party . For example, from midnight to 6 am , the device would
personal call response process 200 awaits such a call. Once 30 not produce any ring tone. Additionally , if desired ,the called
the decision 202 determines that there is an incoming voice party can also configure the electronic device to automati
call, a decision 204 determines whether the incoming voice cally provide an audio message or a text message to the
call is to be answered . Typically, the user of the electronic calling party (e.g., I'm asleep call me tomorrow ” ).
device would signal the electronic device as to whether or FIG . 8 is a flow diagram of an audio message response
not to answer the incoming voice call. Alternatively, the 35 process 300 according to one embodiment of the invention .
electronic device could automatically decide whether to The audio message response process 300 is, for example,
answer the call. suitable for use as the processing carried out by block 214
When the decision 204 determines that the user desires illustrated in FIG . 7 .
the incoming voice call to be answered , the incoming voice The audio message response process 300 initially answers
call is answered 206 and the user engages 208 in a voice call 40 302 the incoming voice call . In this operation , the incoming
with the calling party . A decision 210 then determines voice call is answered 302 but not in a traditional way .
whether the call has ended . When the decision 210 deter Instead , the electronic circuitry associated with a mobile
mines that the call has not yet ended, then the personal call communication device (e.g.,mobile telephone ) that receives
response process 200 can return to repeat the block 208 the incoming voice call operates to answer the incoming
while the voice call continues. Once the decision 210 45 voice call for purposes of an audio message response . For
determines that the voice call has ended , then the personal example , a voice channel is established between the calling
call response process 200 ends. party and the mobile communication device , but the speaker
When the decision 204 determines that the user does not and microphone of the mobile communication device are
desire to answer the incoming voice call, a decision 212 disabled . In effect , in such an embodiment, neither the called
determines whether the user desires to provide an audio 50 party nor the calling party perceives that the voice calling
message to the calling party . When the decision 212 deter has been answered .
mines that the user does desire to provide an audio message Next, one or more predetermined audio messages can be
to the calling party , an audio message is obtained and sent presented 304 by the mobile communication device . The
214 to the calling party ( caller ). presentation 304 of the one or more predetermined audio
Alternatively , when the decision 212 determines that the 55 messages can , for example , be achieved by audio or visual
user does not desire to provide an audio message, a decision means. For example , the predetermined audio messages can
216 determines whether the user desires to provide a text be audio output to a speaker associated with the mobile
message to the calling party . When the decision 216 deter communication device for the called party or can be visual
mines that the user desires to provide a textmessage to the output (e.g., text) to a display of the mobile communication
calling party, a text message is obtained and sent 218 to the 60 device for the called party (e.g., user of the mobile commu
calling party . nication device ).
Still further, when the decision 216 determines that the A decision 306 then determines whether a predetermined
user does not desire to provide a text message to the calling audio message has been selected . Here, the decision 306
party, a decision 220 determines whether the incoming voice determines whether the user ( i.e., called party ) of themobile
call is to be directed to voice mail. When the decision 220 65 communication device has selected one or more of the
determines that the incoming voice call should be directed to predetermined audio messages . When the decision 306
voice mail, then the incoming voice call is directed 222 to determines that a predetermined audio message has been
US 10,492,038 B2
11 12
selected , then the selected audio message is played 308 for selection of a predetermined audio message. Further, in still
the calling party . Here , the mobile communication device other embodiments , an audio message response process can
can output the selected audio message to the calling party first determine whether a custom audio message is to be
over the voice channel. Typically , the mobile communica provided before presenting predetermined audio messages.
tion device of the called party would not produce an audible 5 In yet other embodiments, an audio message response pro
output at the mobile communication device , so that the cess can answer the incoming voice call later in the pro
called party would not be disturbed by the sending of the cessing than operation 302 as shown in FIG . 8 (e.g., before
audio response . The predetermined audio messages are operations 308 and 314 ).
normally short messages (e.g. , not more than 160 characters ) FIG . 9 is a flow diagram of a text message response
so that the duration of time the voice channel is needed 10 process 400 according to one embodiment of the invention .
and / or the amount of network bandwidth consumed is The text message response process 400 is, for example,
minimal. processing performed by the block 218 illustrated in FIG . 7 .
On the other hand , when the decision 306 determines that The text message response process 400 initially drops 402
none of the predetermined audio messages have been the incoming voice call . Here , the information to be supplied
selected , then a decision 310 determines whether a custom 15 to the calling party is a short text message ; therefore , there
audio message is requested . A custom audio message is an is no need for a voice channel.
audio message that is specifically provided for the calling Next, one or more predetermined text messages are dis
party . When the decision 310 determines that a custom audio played 404. Here , the one or more predetermined text
message is not being requested , then the audio message messages would normally be displayed on a display screen
response process 300 returns to repeat the decision 306 and 20 associated with the mobile communication device being
subsequent operations. Alternatively, when the decision 310 utilized by the called party . A decision 406 then determines
determines that a custom audio message is requested , then a whether one (or more ) of the predetermined text messages
custom audio message is recorded 312. Thereafter, the has been selected . When the decision 406 determines that a
custom audio message that has been recorded can be played predetermined text message has been selected , then the
314 for the calling party (caller ). Here, typically , the custom 25 selected text message is transmitted 408 to the caller (i.e.,
audio message would be output by the mobile communica the calling party ) .
tion device of the called party over the voice channel to the On the other hand , when the decision 406 determines that
calling party. Typically, themobile communication device of a predetermined text message has not been selected , then a
mobile communication device, so that the called party 30 requested . When the decision 410 determines that a custom
would not be disturbed by the sending of the audio response . text message is not requested , then the text message
sages ( e.g., not more than 160 characters) so that the subsequent operations. Alternatively , when the decision 410
duration of time the voice channel is needed and /or the determines that a custom text message is requested , then the
amount of network bandwidth consumed is minimal. 35 custom text message is entered 412. Here, the called party
Following the operations 308 and 314 , the incoming voice interacts with themobile communication device to enter the
call is closed 316. In other words, after the selected audio custom text message. Then , the custom text message is
message or the custom audio message is played 308 , 314 , the transmitted 414 to the caller. In one embodiment, the trans
incoming voice call can be closed 316. Following the block mission 408,414 of the textmessage can be performed over
316, the audio message response process 300 is complete 40 a communication network, such as a network having a Short
and ends. Message Service Center (SMSC ) supporting Short Message
to a called party can be determined in a static or dynamic the selected text message or the transmission 414 of the
manner. A static determination would , for example , be when custom textmessage, the textmessage response process 400
the called party has previously set or recorded an audio 45 is complete and ends .
message to be utilized . Typically, with static determination , An alternative embodiment of a text message response
the list of audio messages remains the same ( i.e., static ) until process could operate to answer the incoming voice call and
changed (e.g., by the called party ). A dynamic determination announce to the caller that a textmessage will be forthcom
would allow the audio messages in the list (or the ordering ing. Then , the incoming voice call could be promptly
of the audio messages in the list) to change without specific 50 dropped . This additional operation could , for example , be
action by the user or the called party. For example , the list used with the text message response process 400 by pro
settings, configuration information , or prior usage . Prior trated in FIG . 9 .
usage can include biasing the list of audio messages such The predetermined textmessages being displayed 404 to
that thosemessages being most often selected appear higher 55 a called party can be determined in a static or dynamic
in the list . The list or ordering of the audio messages can also manner. A static determination would , for example , be a text
depend on the calling party, type ofcalling party, location of message the called party has previously set or entered .
calling party or called party , and the like . The list of audio Typically, with static determination , the list of text messages
messages can be represented by text and /or graphics (e.g., remains the same (i.e., static ) until changed (e.g., by the
icons ). 60 called party ). A dynamic determination would allow the text
The audio message response process 300 flexibly enables messages in the list (or the ordering of the text messages in
a user to either select one or more predetermined audio the list) to change automatically, and not by the user. For
messages or provide a custom audio message to be used as example ,the list or ordering of the textmessages can depend
an audio message response to a calling party . However, it on preference settings , configuration information , or prior
should be recognized that, in other embodiments , an audio 65 usage. To illustrate , prior usage can include biasing the list
message response process can alternatively simply pertain to of textmessages such that those messages being most often
US 10,492,038 B2
13 14
text messages can also depend on the calling party, type of this case , the incoming call is dropped , and the entered text
calling party , location of calling party or called party, and the message or the selected one of the predetermined text
like. The list of textmessages can identify each textmessage messages is sent. Examples of text messages are : (i) “ Will
with text (e.g. , at least a portion of the corresponding text call in 10 minutes,” (ii) “ Cannot talk now ,” ( iii) “ I'm in a
message , or an abbreviation ) and / or graphics (e.g. , icons). 5 meeting ," or (iv ) “ Please don't call anymore .” The text
The textmessage response process 400 flexibly enables a messages can be English (or other language ) words or
user to either select one or more predetermined text mes phrases, or can be condensed text strings (e.g., such as slang
sages or provide a custom text message to be used as a text or chat language). In one embodiment, the predetermined
message response to a calling party. However, it should be text messages presented to Tom can be dependent on some
recognized that, in other embodiments , a text message 10 criteria (i.e., automatically selected ). Alternatively, it is
response process can alternatively simply pertain to only possible that Tom might want to edit the predetermined text
providing a custom text message, or only permitting selec message , such can be permitted . As yet another example , the
tion of a predetermined textmessage . Further, in still other text message can embed dynamic information , such as
embodiments, a text message response process can first position , e.g., “ I'm in [ position ] now , so I'll get back to you
determine whether a custom text message is to be provided 15 later .” The position can be determined using a GPS receiver
before presenting predetermined text messages. in the mobile phone or acquired by a remote computer and
FIG . 10 is a flow diagram of an automated call response provided to the mobile phone . The position may also be
process 500 according to one embodiment of the invention. further processed ( locally or remotely) into a more user
The automatic call response process 500 is substantially friendly form , such as city, school, restaurant name, or street
similar in many ways to the personal call response process 20 type addresses. The position could also be used above to
200 illustrated in FIG . 7. However, the automated call assist the user in deciding whether to answer the incoming
response process 500 operates to reduce user input at the call or decline to answer the call .
mobile communication device by making use of stored data 7. If hardware components, configuration or preferences
pertaining to its hardware components, configuration or are taken into consideration , as illustrated in FIG . 10 , the
preferences. In this regard, the automatic call response 25 above scenario can be modified . For example, if Tom is
process 500 includes a decision 502 that determines whether using a head -set with his mobile phone, then an audio
a head - set is active . When the decision 502 determines that message may be most convenient, assuming that Tom wants
a head -set is active , then the automatic call response process to provide a particular (i.e., customized ) message to Bill .
500 can prefer, suggest or require the user to obtain and send The head - set allows Tom to record a brief audio message .
214 an audio message to the caller in response to an 30 Less conveniently , the head - set can be used to present a list
incoming voice call. Alternatively , when the decision 502 of predetermined audio messages and allow Tom's selection
determines that a head -set is not active, then a decision 504 therefrom by a button or voice -command .
can determine whether a display is present. In other words, 8. If Tom is not using a head-set, then a text message
the decision 504 can determine whether the mobile com response might be more suitable . This would typically
munication device has a display. When the decision 504 35 require that Tom's mobile phone have a display and a
determines that the mobile communication device does have keypad . Even so , without a head -set , Tom could still record
a display, then themobile communication device can operate an audio message, though such would likely be less conve
to obtain and send 218 a text message to the caller . Of nient.
course , this assumes that the caller can support text mes 9. Tom can also not provide an audio message or a text
sages even though they initially called with a voice call. 40 message and simply let the incoming call roll-over into
Hence , in another embodiment, the automatic call response voice mail.
process can operate to query or obtain information regarding The exemplary scenario can also be used in a case where
the caller's communication device capabilities. the called party is using one line but the mobile device has
An exemplary scenario of how the previously described multi -line capabilities or call waiting . In such case, the
automatic call response process could work according to one 45 mobile phone can enable the called party to provide a brief
implementation is as follows: audio or text message to the calling party as noted above.
1. From his mobile phone , Bill calls Tom'smobile phone . Alternatively, themobile phone can itself automatically ( i.e.,
2. Tom is alerted by his mobile phone of an incoming call. without user input) respond to the calling party via an audio
Optionally , caller information ( i.e., pertaining to Bill ) can be or text message since the mobile phone is aware that the
displayed or announced to Tom . 50 called party is on the other line .
3. Tom can choose to answer the incoming call or decline In this aspect of the invention , the calling party and the
to answer the call . called party often use mobile communication devices, such
4. In the event that Tom declines to answer the call , Tom asmobile phones. However, the parties can alternatively use
can have the opportunity to provide the caller with a brief other electronic devices , such as a PDA , a computer , etc.
audio or textmessage . 55 Further, the option to provide a text response could be
5. If an audio message is to be provided , then Tom can prevented if the caller's device is known to not support text
either record a personalized message or select one of a messages .
plurality of predetermined audio messages . In this case , the The advantages of the previously described embodiments
incoming call is answered by Tom's mobile phone and then are numerous. Different embodiments or implementations
the audio message is played for the caller, thereafter the call 60 may yield different advantages . One advantage is that com
is dropped . The audio messages are typically brief ( i.e., munications for users of electronic devices can be flexibly
short), and examples of audio messages are: (i) “Will call in provided . Another advantage is that communication mode
10 minutes,” ( ii ) “ Cannot talk now ,” ( iii ) “ I'm in a meeting ,” changes can be performed at an electronic device to better
22
or (iv ) “ Please don't call anymore.” suit the needs or condition of the electronic device or user
6.On the other hand, if a textmessage is to be provided , 65 preferences. In still another advantage, a user can provide
then Tom can either enter a personalized text message or feedback to a caller without answering a voice call from the
select from a plurality of predetermined text messages . In caller.
US 10,492,038 B2
15 16
Another aspect of the invention pertains to improved mobile communication device . Following the blocks 608
approaches to respond to an incoming text message . The and 610 , the message presentation process 600 ends.
improved approaches enable a recipient to provide a reply As discussed above, text-to -speech conversion can be
message to an initiator. The incoming text message can be invoked and performed on an electronic device, which may
presented to the recipient with an audio or textual presen- 5 be a mobile communication device. While text-to -speech
tation . Thereafter, a reply text message can be sent back to conversion , particularly if high quality is desired , requires
the initiator. The recipient can form the reply text message substantial processing capabilities , mobile electronic
by recording a brief audio message or entering a text devices , such as mobile communication devices, given their
message . In the case in which a brief audio message is used , small form
the audio message can be automatically converted to a text 10 processing factor and price competition , tend to have limited
capability . Accordingly, in one embodiment,
message before being transmitted to the initiator. text - to -speech conversion can be off -loaded from the mobile
FIG . 11 is a flow diagram of a message presentation device . For example , a remote server computer can be
process 600 according to one embodiment of the invention . provided the textmessage and produce the resulting audio
The message presentation process 600 is performed by an
electronic device , such as a mobile communication device. 15 message , and then supply the audio message to the mobile
The message presentation process 600 begins with a device . The remote server computer can be a networked
decision 602 that determines whether an incoming text server coupled to the network 108. One example of a
message is present. Typically, the incoming text message networked server is a gateway computer for a wireless
would be transmitted to the mobile communication device electronic device , such as a mobile telephone .
from another communication device . When the decision 602 20 FIG . 12 is a flow diagram of a reply message process 700
determines that an incoming text message is not present, according to one embodiment of the invention . The reply
then the message presentation process 600 awaits such message process 700 is performed by an electronic device ,
message . Once the decision 602 determines that an incom such as a mobile communication device .
ing text message has been received , a decision 604 deter The reply message process 700 begins with a decision 702
mines whether an audio or text presentation is to be utilized . 25 that determines whether a reply message is to be sent.
The decision 604 can be performed in a variety of different Typically, the reply message process 700 follows the pre
ways. For example , the determination of whether to utilize sentation of an incoming textmessage to a user of a mobile
an audio or text presentation can be based on user input or communication device . Hence, the reply message to be sent
can be automatically determined through a use of configu is a reply to the incoming text message . However , in other
ration or preference information or hardware components 30 embodiments, the reply message to be sent can be merely an
( e.g., display, speaker, head - set). initial message as opposed to a response to an earlier
When the decision 604 determines that an audio presen message .
tation is to be ut ed , the incoming text message is con In any case , when the decision 702 determines that a reply
verted 606 to an audio message. For example , a text-to message is not to be sent, then the reply message process 700
speech conversion can be performed . In one embodiment, a 35 ends or simply awaits the need to send a reply message. On
user of the electronic device can be permitted to choose the other hand, when the decision 702 determines that a
speech characteristics, such as a voice, tone, pace , accent, or reply message is to be sent, then a decision 704 determines
mood , for the resulting speech . For example, a user could whether an audio or text message is to be formed . The
choose speech characteristics by preference settings. In decision 704 can be performed in a variety ofdifferent ways.
another embodiment, the incoming text message can include 40 For example, the determination of whether to send an audio
or reference speech characteristics so that the initiator can or text message can be based on user input or can be
control or influence speech characteristics . In still another automatically determined through a use of configuration or
embodiment, if the text to be converted contains condensed preference information or hardware components (e.g., dis
text (e.g., such as slang or chat language), the resulting play, speaker, head - set).
speech can pertain to an uncondensed form of the text. The 45 When the decision 704 determines that an audio message
ability to convert from condensed text to resulting speech for is to be formed , then the reply message process 700 prompts
uncondensed text can be facilitated by pattern matching . For 706 for an audio message . Here, the prompt 706 can be
example , in chat language “ LOL” can be converted to an directed to the user of the mobile communication device .
audio message for “ lots of love ." In one implementation , a The prompt can be an audio or textual indication . Next, a
table can store audio messages corresponding to chat terms 50 decision 708 determines whether an audio message has been
or phrases. In another implementation , a first table would recorded . When the decision 708 determines that the audio
store uncompressed terms or phrases corresponding to chat message has not been recorded , then the reply message
terms or phrases, and a second table would store audio process 700 awaits the audio message. Once the decision
messages corresponding to the uncompressed terms or 708 determines that the audio message has been recorded ,
phrases. 55 then the audio message is converted 710 to a text message .
After the incoming text message is converted to the audio In one embodiment, if the audio message recorded is greater
message , the audio message is played 608. Typically, the than a maximum text message size ( e.g., 150 or 160 char
audio message is played 608 by the mobile communication acters), then the audio message can be shortened so that the
device for the user. For example, the audio message can be resulting text message does not exceed the maximum text
output to a speaker of the mobile communication device or 60 message size. One way to shorten the textmessage is to use
a headset used therewith . As a result, the user of themobile abbreviations. For example, the words “ For example” can be
wireless communication device receives an audio message changed to “ e.g. ” . Such conversion can be again be per
even though the incoming message was a text message . formed by matching entries in tables . Another way to
On the other hand , when the decision 604 determines that shorten is to remove non -essential text. Still another way to
a text presentation is to be utilized , the incoming text 65 shorten is to clip off or truncate the text message at the
message is displayed 610. Here, the incoming textmessage maximum text message size . In another embodiment, the
would be displayed 610 on a display associated with the resulting text message might provide an indication that it
US 10,492,038 B2
17 18
was converted from an audio message . Following the block duce and send a reply message based on user settings or
710 , the text message is transmitted 712 over a wireless preferences, position , configuration , status, etc.
network . In this aspect of the invention , the calling party and the
Alternatively , when the decision 704 determines that a called party often use mobile communication devices, such
text message is to be formed , then a text entry screen is 5 asmobile phones. However, the parties can alternatively use
displayed 714. Next, a decision 716 determines whether a other electronic devices, such as a PDA, a computer, etc.
text message has been entered . When the decision 716 The advantages of the invention exemplified by FIGS .
11-12 are numerous. Different embodiments or implemen
determines that a textmessage has not yet been entered , then tationsmay
the reply message process 700 awaits entry of the text 10 communications yield different advantages. One advantage is that
message. Once the text message has been entered , the text flexibly provided . for users of electronic devices can be
Another advantage is that communication
message is transmitted 712 over the wireless network . mode changes can be performed
Following the block 712 , the reply message process 700 better suit the needs or condition ofat the an electronic device to
electronic device or
ends. user preferences . Still another advantage is that conversion
Although the reply message process 700 provides for the 15 of an audio message to a text message facilitates use a low
user to enter a custom text or audio message , it should be cost network ( such as the SMS network ). Another advantage
understood that the reply message can alternatively be is reduced network bandwidth load . Yet still another advan
formed through use of semi-custom or predetermined reply tage is that the sender can get back a message in the same
messages from which the user of the mobile communication format as they sent the original message , though the recipi
device can choose . The use of semi-custom or predeter- 20 entmay use the message in a different format or mode ( e.g.,
mined reply messages can be achieved as noted above in a recipient hears the textmessage as an audio message).
number of embodiments , and can serve to simplify the Moreover, it should be noted that with regards to any of
An exemplary scenario of how message presentation and incoming to an electronic device, notonly can the user of the
reply message processes could work according to one imple- 25 mobile device take an action (e.g., button press or voice
mentation of the second aspect is as follows: command ) to decline the call /message but also the electronic
1. From his mobile phone, Bill prepares and sends a text device itself can automatically decline the call/message such
message to Tom's mobile phone . that the user is not disturbed . For example , an electronic
2. Tom is alerted by his mobile phone of an incoming text device can be configured through user settings (e.g., pref
message, such as by displaying at least a portion of the text 30 erences) to decline calls /messages matching certain criteria .
message and/or otherwise notifying Tom of the text mes Also , an auto reply message can be configured to be auto
sage . matically sent in response to the call /message. For a known,
3. Tom's mobile phone can decide whether to present the undesired marketing caller /message sender, the electronic
textmessage on a display screen of Tom's mobile phone, or device can automatically send a reply message demanding
to first convert the text message to an audio message and 35 the sender not to call or send messages anymore , and to
then present the audio message to Tom (e.g. , play the audio remove your information from their database .
message ). Of course , Tom can interact with Tom's mobile Text messages received or sent can optionally embed
present the message . text message be converted to an audio format. The speech
4. Thereafter , if desired , Tom can prepare and send a reply 40 characteristics can pertain to voice, tone , pace , accent,
message back to Bill. This reply message can be prepared and /or mood . The speech characteristics for the resulting
initially as a text message or an audio message . Tom's speech can be set in preference or configuration information ,
mobile phone and/or Tom can determine whether the reply set on a per message basis by users, or set by evaluation of
message is initially prepared as a text message or as an audio monitored data pertaining to the user.
message . If an audio message is initially created , such audio 45 Additionally, the messages being transmitted can be
message must be converted to a text message prior to encrypted for security purposes.
transmission . Eventually, the reply message is sent to Bill as In one embodiment, an electronic device performing
a text message . Tom's mobile phone can assist with the communications using audio and/or textmessages according
custom or predetermined reply message from which Tom 50 thereto ) one or more electrical components for enhancing
and/or Tom's mobile phone can choose . the hearing of the user of the electronic device . The elec
5. If Tom is using a head -set with his mobile phone, then tronic device will normally include a microphone and a
Tom wants to provide a particular ( i.e. , customized )message an automated secretary for a user of an electronic device .
to Bill. The head -set allows Tom to easily record a brief 55 The automated secretary can completely or partially respond
audio message. Less conveniently , the head - set can be used to an incoming call/message so as to reduce disturbances to
to present a list of predetermined audio messages and allow the user . The user can personalize the automated secretary
Tom's selection therefrom by a button or voice -command. through user settings (e.g., preferences ), or the automated
6. If Tom is not using a head -set, then a text message secretary can learn over time how to handle different incom
response might be more suitable . This would typically 60 ing calls/messages. Besides handling or assisting the user
require that Tom's mobile phone have a display and a with incoming calls /messages, the automated secretary can
keypad . Even so , without a head -set, Tom could still record also assist with other activities, such as making calendar
an audio message, though such would likely be less conve entries (e.g. , meetings) in a calendar or responding to
7. Tom can also not provide a reply message and simply 65 taining to the user's schedule as maintained by the calendar
not respond to the incoming text message. Alternatively , (though the user could restrict such access to certain infor
Tom can configure his mobile phone to automatically pro mation and /or inquiring parties ). For example , if an incom
US 10,492,038 B2
19 20
ing text message asks “ available for lunch today ?" , the The many features and advantages of the present inven
automated secretary can check the user's availability for tion are apparent from the written description and , thus, it is
lunch by way of the user's calendar , then if the user is not intended by the appended claimsto cover all such features
available the automated secretary can quickly informing the and advantages of the invention . Further , since numerous
inquiring party of same or propose another date. On the 5 modifications and changes will readily occur to those skilled
other hand , if the lunch time period is available in the user's in the art, it is not desired to limit the invention to the exact
calendar, then the automated secretary can either directly construction and operation as illustrated and described .
respond to the inquiring party of acceptance or propose a Hence , all suitable modifications and equivalents may be
response to the user for review , modification and/ or trans resorted to as falling within the scope of the invention .
mission . 10
It should be obvious to those skilled in the art that a What is claimed is :
number of embodiments performing communications using 1. An article for managing messaging of a plurality of
voice as well as audio and /or text messages can be imple
mented using voice over Internet Protocol technologies , users using at least a network -based portal at least based on
with signals delivered over the Web . For example, a calling 15 Internet protocol, with different communication modes
party's communication or mobile device can include an allowed , depending on each of the plurality of users having
adapter to convert voice signals to data packets before an identifier for use with the different communication
sending them over the Internet. A service provider can modes, with the corresponding identifier being set via the
convert the packets back into voice signals before sending network -based portal, and without requiring the plurality of
the voice signals to the called party's communication 20 users to disclose their contact information to each other, the
device. Similarly , embodiments can be implemented using article comprising:
voice over wireless protocols , such as Wi-Fi or Wi-Max a non - transitory computer readable storagemedium com
networks. Using such technologies, computing devices can prising a plurality of instructions, when executed by at
become communication devices. least a server, result in the at least a server :
The various embodiments, implementations, features and 25 providing a plurality of communication modes to a first
aspects of the invention noted above can be combined in user to allow the first user to use one of the plurality
various ways or used separately. Those skilled in the art will of communication modes as a selected communica
understand from the description that the invention can be tion mode for a message from the first user to a
equally applied to or used in other different settings with second user via an electronic device associated with
respect to various combinations, embodiments , implemen- 30 the second user, with the first user being identified at
tations or features provided in the description herein . least depending on a prior registration process by the
The invention can be implemented in software, hardware first user regarding the use of the network - based
or a combination of hardware and software . A number of portal,
embodiments of the invention can also be embodied as wherein the plurality of communication modes
computer readable code on a computer readable medium . 35 include at least text communication using a per
The computer readable medium is any data storage device sonal computer, voice communication using a
that can store data which can thereafter be read by a personal computer , text communication using a
computer system . Examples of the computer readable mobile phone , voice communication using a
medium include read -only memory, random - access mobile phone, and communication with at least an
memory, CD -ROMs, magnetic tape , optical data storage 40 image , and
devices, and carrier waves. The computer readable medium wherein messages are eligible to be received by the
can also be distributed over network -coupled computer electronic device associated with the second user
systems so that the computer readable code is stored and based on any of the plurality of communication
executed in a distributed fashion . modes , all depending on an identifier associated
Numerous specific details are set forth in order to provide 45 with the second user being set by the second user
a thorough understanding of the present invention .However, via the network -based portal, at least in view of
it will become obvious to those skilled in the art that the the network - based portal based on the Internet
invention may be practiced without these specific details. protocol;
The description and representation herein are the common receiving an urgency indication regarding the message ,
meanings used by those experienced or skilled in the art to 50 wherein the urgency indication being from the first
most effectively convey the substance of their work to others user via an electronic device associated with the first
skilled in the art. In other instances , well -known methods, user; and
procedures, components , and circuitry have not been enabling , via the network -based portal, the message to
described in detail to avoid unnecessarily obscuring aspects be received by the second user through the electronic
of the present invention . 55 device associated with the second user , using the
In the foregoing description, reference to " one embodi selected communication mode, based on the identi
ment” or “ an embodiment” means that a particular feature, fier associated with the second user, in view of the
structure , or characteristic described in connection with the second user not blocking the first user from using the
embodiment can be included in at least one embodiment of selected communication mode to reach the second
the invention . The appearances of the phrase “ in one 60 user, via the network -based portal, wherein the plu
embodiment” in various places in the specification are not rality of instructions, when executed by the at least
necessarily all referring to the same embodiment, nor are a server,
separate or alternative embodiments mutually exclusive of further result in the at least a server:
other embodiments . Further, the order of blocks in process receiving , from the second user, contact information
flowcharts or diagrams representing one or more embodi- 65 associated with the second user to allow the second
ments of the invention do not inherently indicate any par user to participate and at least receive messages via
ticular order nor imply any limitations in the invention . the network -based portal;
US 10,492,038 B2
21 22
determining availability of the second user ; and registration process by the first user regarding the use
enabling a notification to reach the second user via the of the network -based portal,
network - based portal, at least in view of receiving wherein the plurality of communication modes include
the urgency indication regarding the message and at at least text communication using a personal com
least in view of the first user not being blocked from 5 puter , voice communication using a personal com
reaching the second user via the network -based puter, text communication using a mobile phone,
portal, with the notification for notifying the second voice communication using a mobile phone, and
user , communication with at least an image , and
wherein even when themessage is received by the second wherein messages are eligible to be received by the
user through the electronic device associated with the 10 electronic device associated with the second user
second user depending on the identifier associated with based on any of the plurality of communication
the second user, the contact information associated with modes, all depending on an identifier associated with
the second user is not provided via the network -based the second user being set by the second user via the
portal to the first user through the electronic device network -based portal, at least in view of the net
associated with the first user, and 15 work -based portal based on the Internet protocol;
wherein the identifier associated with the second user is computer program code for permitting the second user to
distinct from the contact information associated with block the first user from using at least the selected
the second user. communication mode to reach the second user via the
2. An article for managing messaging as recited in claim network -based portal;
1, wherein the notification is enabled to be generated at least 20 computer program code for enabling , via the network
in an audio manner. based portal, the first message to be received by the
3. An article for managing messaging as recited in claim second user via the electronic device associated with
2 , wherein the plurality of instructions, when executed by the second user, using the selected communication
the at least a server, further result in the at least a server not mode , depending on the identifier associated with the
interrupting the second user by the message unless overrid- 25 second user, in view of the second user not blocking the
den based on the urgency indication regarding the message . first user from using the selected communicationmode
4. An article for managing messaging as recited in claim to reach the second user, via the network -based portal;
3 , wherein the plurality of instructions, when executed by computer program code for determining availability of the
the at least a server, further result in the at least a server second user; and
receiving data regarding a setting digitally set by the second 30 computer program code for receiving , from the second
user at the electronic device associated with the second user , user, contact information associated with the second
wherein the second user would not be interrupted by the user to allow the second user to participate and at least
message in view of the received data regarding the setting, receive messages via the network -based portal,
unless overridden based on the urgency indication regarding wherein even when the first message is received by the
the message. 35 second user via the electronic device associated with
5. An article for managing messaging as recited in claim the second user depending on the identifier associated
1 , wherein the plurality of instructions, when executed by with the second user, the contact information associated
the at least a server, further result in the at least a server not with the second user is not provided via the network
interrupting the second user by the message unless overrid based portal to the first user via an electronic device
den based on the urgency indication regarding the message . 40 associated with the first user, and
6. An article formanaging messaging as recited in claim wherein the identifier associated with the second user is
1 , wherein the plurality of instructions, when executed by distinct from the contact information associated with
the at least a server, further result in the at least a server the second user.
receiving data regarding a setting digitally set by the second 8. A non -transitory computer readable medium as recited
user at the electronic device associated with the second user, 45 in claim 7 , wherein said computer readable medium com
wherein the second user would not be interrupted by the prises computer program code for:
message in view of the received data regarding the setting, receiving an urgency indication from the first user to the
unless overridden based on the urgency indication regarding second user via the electronic device associated with
the message. the first user ; and
7. A non - transitory computer readable medium including 50 enabling an urgent notification to be generated to notify
at least executable computer program code stored therein for the second user at least in view of receiving the urgency
managing electronic communications of a plurality of users indication .
using at least a network -based portal at least based on 9. A non - transitory computer readable medium as recited
Internet protocol, with different communication modes in claim 8, wherein the urgent notification is enabled to be
allowed , depending on each of the plurality of users having 55 generated at least in an audio manner.
an identifier for use with the different communication 10. A non -transitory computer readable medium as recited
modes, with the corresponding identifier being set via the in claim 7 , wherein the identifier associated with the second
network -based portal, and without requiring the plurality of user includes a digital identity of the second user.
users to disclose their contact information to each other, said 11. A non - transitory computer readable medium as recited
computer readable medium comprising: 60 in claim 7 , wherein the contact information associated with
computer program code for providing a plurality of com the second user allows the second user to be electronically
munication modes to a first user to allow the first user contacted based on the contact information , without using
to use one of the plurality of communication modes as the network - based portal.
a selected communication mode for a first message 12. A non -transitory computer readable medium as recited
from the first user to a second user via an electronic 65 in claim 7 ,
device associated with the second user, with the first wherein the availability of the second user is related to
user being identified at least depending on a prior receiving messages ,
US 10,492,038 B2
23 24
wherein the electronic device associated with the second into an audio message so that the first message is
user is a mobile phone , and received by the second user in an audio manner via a
wherein the electronic device associated with the first user speaker if the first user is not blocked from reaching the
is a mobile phone . second user.
13. A non -transitory computer readable medium as recited 5 22. A non - transitory computer readable medium as recited
in claim 7, wherein said computer readable medium com in claim 7 ,
prises computer program code for receiving an audio mes wherein the identifier associated with the second user
sage from the second user after the second user has received includes a digital identity of the second user , and
the first message, if the first user is not blocked from wherein the contact information associated with the sec
reaching the second user , and transforming the audio mes 10 ond user allows the second user to be electronically
sage into a text message to be received by the first user. contacted based on the contact information , without
14. A non -transitory computer readable medium as recited using the network -based portal.
in claim 13, 23. A non -transitory computer readable medium as recited
wherein the first message is a text message from the first in claim 22, wherein the contact information associated with
user, and 15 the second user includes at least one of a phone number or
wherein said computer readable medium comprises com an email address of the second user.
puter program code for transforming the text message 24. A non -transitory computer readable medium as recited
from the first user into an audio message so that the text in claim 23 ,
message from the first user is received by the second wherein the electronic device associated with the second
user in an audio manner via a speaker if the first user 20 user is a mobile phone, and
is not blocked from reaching the second user. wherein the electronic device associated with the first user
15. A non -transitory computer readable medium as recited is a mobile phone.
in claim 14 , 25. A non -transitory computer readable medium as recited
wherein the electronic device associated with the second in claim 24 , wherein said computer readable medium com
user is a mobile phone , and 25 prises computer program code for receiving an audio mes
wherein the electronic device associated with the first user sage from the second user after the second user has received
is a mobile phone. the first message , if the first user is not blocked from
16. A non - transitory computer readable medium as recited reaching the second user , and transforming the audio mes
in claim 13, wherein the contact information associated with sage into a text message to be received by the first user.
the second user allows the second user to be electronically 30 26. A non -transitory computer readable medium as recited
contacted based on the contact information , without using in claim 25 , wherein the firstmessage is a textmessage from
the network -based portal. the first user.
17. A non -transitory computer readable medium as recited 27. A non -transitory computer readable medium as recited
in claim 16 , in claim 26 , wherein said computer readable medium com
wherein the first message is a text message from the first 35 prises computer program code for transforming the text
user, and message from the first user into an audio message so that the
wherein said computer readable medium comprises com text message from the first user is received by the second
puter program code for transforming the textmessage user in an audio manner via a speaker if the first user is not
into an audio message so that the text message is blocked from reaching the second user.
received by the second user in an audio manner via a 40 28. A non - transitory computer readable medium as recited
speaker if the first user is notblocked from reaching the in claim 25 ,
second user. wherein the first message is not an audio message , and
18. A non -transitory computer readable medium as recited wherein said computer readable medium comprises com
in claim 16 , wherein said computer readable medium com puter program code for transforming the first message
prises computer program code for: 45 into an audio message so that the first message is
receiving an urgency indication for the second user, with received by the second user in an audio manner via a
the urgency indication being from the first user; and speaker if the first user is not blocked from reaching the
enabling an urgent notification to be generated to notify second user.
the second user at least in view of receiving the urgency 29. A non - transitory computer readable medium as recited
indication from the first user to the second user. 50 in claim 28 , wherein said computer readable medium com
19. A non -transitory computer readable medium as recited prises computer program code for:
in claim 18 , wherein the urgent notification is enabled to be receiving a notification indication from the second user
generated at least in an audio manner. via the electronic device associated with the second
20. A non - transitory computer readable medium as recited user, with the notification indication regarding another
in claim 7 , 55 message ;
wherein the first message is a text message , and enabling a notification to reach the first user via the
wherein said computer readable medium comprises com network - based portal, at least in view of receiving the
puter program code for transforming the textmessage notification indication ; and
into an audio message so that the text message is not interrupting the first user by the another message,
received by the second user in an audio manner via a 60 unless overridden based on the notification indication
speaker if the first user is not blocked from reaching the regarding the another message .
second user. 30. A non - transitory computer readable medium as recited
21. A non -transitory computer readable medium as recited in claim 29 , wherein the notification at least conveys
in claim 7 , urgency.
wherein the first message is not an audio message , and 65 31. A non -transitory computer readable medium as recited
wherein said computer readable medium comprises com in claim 23 , wherein said computer readable medium com
puter program code for transforming the first message prises computer program code for:
US 10,492,038 B2
25 26
receiving a notification indication from the second user 38. A computer -implemented method for managing elec
via the electronic device associated with the second tronic communications of a plurality of users using at least
user, with the notification indication regarding another a network -based portal at least based on Internet protocol,
message; with different communication modes allowed , depending on
enabling a notification to reach the first user via the 5 each of the plurality ofusers having an identifier for use with
network -based portal, at least in view of receiving the the different communication modes, with the corresponding
notification indication ; and identifier being set via the network - based portal, and without
not interrupting the first user by the another message , requiring the plurality of users to disclose their contact
unless overridden based on the notification indication information to each other, said method comprising :
regarding the another message . 10
32. A non -transitory computer readable medium as recited providing a plurality of communication modes to a first
in claim 31 , wherein the notification at least conveys user to allow the first user to use one of the plurality of
urgency . communication modes as a selected communication
33. A non -transitory computer readable medium as recited mode for a firstmessage from the first user to a second
in claim 7, wherein said computer readable medium com- 15 user via an electronic device associated with the second
prises computer program code for receiving a second mes user, with the first user being identified at least depend
sage from the second user to the first user, wherein one of the ing on a prior registration process by the first user
first message or the second message is voice and the other regarding the use of the network -based portal,
one of the first message or the second message is text. wherein the plurality of communication modes include
34. A non -transitory computer readable medium as recited 20 at least text communication using a mobile phone ,
in claim 7 , voice communication using a mobile phone, and
wherein the enabling the first message to be received by communication with at least an image, and
the second user depends on an identifier associated with wherein messages are eligible to be received by the
the first user, electronic device associated with the second user
wherein the identifier associated with the second user is a 25 based on any of the plurality of communication
digital identity of the second user, modes, all depending on an identifier associated with
wherein the identifier associated with the first user is a the second user being set by the second user via the
digital identity of the first user, and network -based portal, at least in view of the net
wherein the contact information associated with the sec work - based portal based on the Internet protocol;
ond user allows the second user to be electronically 30 permitting the second user to block the first user from
contacted based on the contact information , without using at least the selected communication mode to
using the network - based portal. reach the second user via the network - based portal;
35. A non -transitory computer readablemedium as recited
in claim 7 , enabling, via the network -based portal , the first message
wherein the enabling the first message to be received by 35 to be received by the second user via the electronic
the second user depends on an identifier associated with device associated with the second user, using the
the first user, selected communication mode , depending on the iden
wherein the contact information associated with the sec tifier associated with the second user, in view of the
ond user allows the second user to be electronically second user not blocking the first user from using the
contacted based on the contact information , without 40 selected communication mode to reach the second user,
using the network -based portal, and via the network -based portal;
wherein said computer readable medium comprises com determining availability of the second user; and
puter program code for receiving a second message receiving , from the second user, contact information asso
from the second user for the first user, wherein one of ciated with the second user to allow the second user to
the first message or the second message is a voice 45 participate and at least receive messages via the net
message, and the other one of the firstmessage or the work -based portal,
second message is a textmessage , thereby allowing the
first user and the second user to communicate using wherein even when the first message is received by the
different communication modes , based on the identifier second user via the electronic device associated with
associated with the first user and the identifier associ- 50 the second user depending on the identifier associated
ated with the second user. with the second user, the contact information associated
36. A non - transitory computer readable medium as recited with the second user is not provided via the network
in claim 35 , based portal to the first user via the electronic device
wherein one of the electronic device associated with the associated with the first user, and
first user or the electronic device associated with the 55 wherein the identifier associated with the second user is
second user is an electronic device with a computer distinct from the contact information associated with
keyboard , and the second user.
wherein the other one of the electronic device associated 39. A computer-implemented method as recited in claim
with the first user or the electronic device associated
with the second user is an electronic device without a 60 38 ,
computer keyboard . wherein the enabling the first message to be received by
37. A non - transitory computer readable medium as recited the second user depends on an identifier associated with
in claim 36 , wherein said computer readable medium com the first user, and
prises computer program code to enable the second user to wherein the contact information associated with the sec
select a predetermined message to send to the first user, the 65 ond user allows the second user to be electronically
predetermined message being selected from a set of prede contacted based on the contact information , without
termined messages provided to the second user. using the network -based portal.
US 10,492,038 B2
27 28
40. A computer- implemented method as recited in claim second user via the network -based portal, which
39, allows the second user to efficiently maintain the
wherein the contact information associated with the sec second user's communication using the plurality of
ond user includes at least one of a phone number or an modes of communication ;
email address of the second user. 5 computer program code for permitting the second user to
41. A computer- implemented method as recited in claim block the first user from using at least the selected mode
40 comprising receiving a secondmessage from the second of communication to communicate with the second
user for the first user, wherein one of the first message or the user via the network -based portal, based on the iden
second message is a voice message , and the other one of the tifier associated with the first user ;
first message or second message is a text message , thereby 10 computer program code for enabling the first message to
allowing the first user and the second user to communicate be electronically provided to the second user, using the
using different communication modes, based on the identi selected mode of communication , depending on the
fier associated with the first user and the identifier associated identifier associated with the second user, in view of the
with the second user. second user not blocking the first user from using the
42. A computer- implemented method as recited in claim 15 selected mode of communication to communicate with
41 comprising enabling the second user to select a prede the second user, via the network -based portal;
termined message to send to the first user, the predetermined computer program code for determining availability of the
message being selected from a set of predetermined mes second user related to receiving messages ; and
sages provided to the second user. computer program code for receiving, from the second
43. A computer-implemented method as recited in claim 20 user, contact information associated with the second
41 comprising receiving an urgency indication from the user to allow the second user to participate and at least
second user to the first user via the electronic device receive messages via the network -based portal,
associated with the second user to generate an urgent noti wherein even when the first message is received by the
fication to notify the first user. second user via the selected mode of communication ,
44. A computer - implemented method as recited in claim 25 the contact information associated with the second user
43 , wherein the urgent notification is enabled to be generated is not provided via the network -based portal to the first
at least in an audio manner . user, and contact information associated with the first
45. A computer-implemented method as recited in claim user is not provided via the network - based portal to the
41 comprising enabling the second user to select a prede second user , so as to provide an option to the second
termined message to send to the first user , the predetermined 30 user to keep the contact information associated with the
message being selected from a set of predetermined mes second user confidential from the first user, and to
sages provided to the second user , provide an option to the first user to keep the contact
wherein an urgency indication is received from the second information associated with the first user confidential
user to the first user via the electronic device associated from the second user, and
with the second user to generate an urgent notification 35 wherein the identifier associated with the second user is
to notify the first user. distinct from the contact information associated with
46. A non -transitory computer readable medium including the second user, and the identifier associated with the
at least executable computer program code stored therein first user is distinct from the contact information asso
that facilitates electronic communication of a plurality of ciated with the first user .
users using at least a network -based portal at least based on 40 47. A non -transitory computer readable medium as recited
Internet protocol, with a plurality ofmodes of communica in claim 46 , wherein the computer readable medium com
tion available for the plurality of users to communicate, with prises:
each of the plurality of users having an identifier for use with computer program code for receiving an indication from
the plurality of modes of communication , and without the second user for the first user via the network -based
requiring the plurality of users to disclose their contact 45 portal to generate an urgent notification for the first
information to each other, the computer readable medium user; and
comprising : computer program code for enabling the urgent notifica
computer program code for providing a plurality of modes tion to be generated to notify the first user at least in
of communication to a first user to allow the first user view of receiving the indication .
to use one of the plurality ofmodes of communication 50 48. A non -transitory computer readable medium as recited
as a selected mode of communication for a first mes in claim 47 , wherein the urgent notification is enabled to be
sage to be sent from the first user to a second user, generated at least in an audio manner.
based on an identifier associated with the first user 49. A non -transitory computer readable medium as recited
previously set by the first user via the network -based in claim 46 , wherein the computer readable medium com
portal, 55 prises :
wherein the plurality ofmodes of communication sup computer program code for determining whether the
ported by the network -based portal include at least second user has blocked the first user from using the
text communication using a personal computer, selected mode of communication to communicate the
voice communication using a personal computer, first message with the second user , via the network
text communication using a mobile phone voice 60 based portal;
communication using a mobile phone, and commu computer program code for receiving a notification indi
nication with at least an image , and cation from the first user to the second user via the
wherein messages are eligible to be received electroni network -based portal, with the notification indication
cally by the second user via the network -based regarding another message ; and
portal, based on any of the plurality of modes of 65 computer program code for enabling the another message
communication , all depending on an identifier asso to be electronically provided to the second user, in view
ciated with the second user previously set by the of the notification indication , even though the second
US 10,492,038 B2
29 30
user has blocked the first user from using the selected 58. A non -transitory computer readable medium as recited
mode of communication to communicate the first mes in claim 46 ,
sage with the second user. wherein said computer readable medium comprises com
50. A non -transitory computer readable medium as recited puter program code for receiving a second message
in claim 49 , wherein the another message conveys at least 5 from the second user to the first user, with one of the
urgency . first message or the second message being voice and
51. A non - transitory computer readable medium as recited the other one of the firstmessage or the second message
in claim 46 , being text, and
wherein the network -based portal allows the first user to 10 wherein ond
the contact information associated with the sec
user includes at least one of a phone number or an
receive messages from the second user via an electronic email address of the second user.
device associated with the second user that has a first 59. A non -transitory computer readable medium as recited
set of buttons for receiving user input, in claim 58 , wherein said computer readable medium com
wherein the network -based portal allows the second user prises computer program code for enabling the second user
to receive messages from the first user via an electronic 15 to select a predetermined message to send to the first user via
device associated with the first user that has a second at least one of the plurality ofmodes of communication , the
set of buttons for receiving user input, and predetermined message being selected from a set of prede
wherein the number of buttons in the first set is different termined messages provided to the second user.
from the number of buttons in the second set. 60. A non -transitory computer readable medium as recited
52. A non - transitory computer readablemedium as recited 20 in claim 59 , the computer readable medium comprises
in claim 46 , computer program code for enabling a message to be
wherein the network -based portal allows the first user to provided to the second user from the first user , with the
receive messages from the second user via an electronic message based on a mode of communication that is different
device associated with the second user that has a from the selected mode, even though the second user has
computer keyboard , and 25 blocked the first user from using the selected mode of
wherein the network -based portal allows the second user communication to communicate the first message with the
to receive messages from the first user via an electronic second user.
device associated with the first user that does not have 61. A non -transitory computer readable medium as recited
a computer keyboard . in claim 60 , wherein the mode of communication different
53. A non -transitory computer readable medium as recited 30 least
from antheimage
selected mode includes communication with at
and communication in an audio manner .
in claim 52 , wherein the computer readable medium com 62. A non -transitory computer readable medium as recited
prises: in claim 61,
computer program code for determining whether the wherein one of the electronic device associated with the
second user has blocked the first user from using the 35 first user or the electronic device associated with the
selected mode of communication to communicate the second user is a personal computer , and
first message with the second user, via the network wherein the other one of the electronic device associated
based portal; with the first user or the electronic device associated
computer program code for receiving a notification indi with the second user is a mobile phone .
cation from the first user to the second user via the 40 63. A non -transitory computer readable medium as recited
network -based portal , with the notification indication in claim 61 , wherein said computer readable medium com
regarding another message; and prises computer program code for permitting the second user
computer program code for enabling the another message to block the first user from using the mode of communica
to be electronically provided to the second user , in view tion different from the selected mode to communicate with
of the notification indication, even though the second 45 the second user via the network -based portal, based on the
user has blocked the first user from using the selected identifier associated with the first user.
mode of communication to communicate the first mes 64. A non -transitory computer readable medium as recited
sage with the second user. in claim 46 ,
54. A non - transitory computer readable medium as recited wherein the contact information associated with the sec
in claim 53 , wherein the another message conveys at least 50 ond user includes at least one of a phone number or an
urgency . email address of the second user , and
55. A non - transitory computer readable medium as recited wherein said computer readable medium comprises com
in claim 46 , wherein said computer readable medium com puter program code for enabling a message to be
prises computer program code for receiving a second mes provided to the second user from the first user, with the
sage from the second user to the first user, wherein one of the 55 message based on a mode of communication that is
first message or the second message is voice and the other different from the selected mode, even though the
of the first message or the second message is text. second user has blocked the first user from using the
56.A non -transitory computer readable medium as recited selected mode of communication to communicate the
in claim 46 , wherein said computer readable medium com first message with the second user .
prises computer program code for enabling the second user 60 65. A non -transitory computer readable medium as recited
to select a predetermined message to send to the first user, in claim 64 , wherein the mode of communication different
the predetermined message being selected from a set of from the selected mode includes communication with at
predetermined messages provided to the second user. least an image and communication in an audio manner.
57. A non - transitory computer readable medium as recited 66. A non -transitory computer readable medium as recited
in claim 46 , wherein the contact information associated with 65 in claim 65 , wherein said computer readable medium com
the second user includes at least one of a phone number or prises computer program code for permitting the second user
an email address of the second user . to block the first user from using the mode of communica
US 10,492,038 B2
31 32
tion different from the selected mode to communicate with from the second user to the first user, and transforming
the second user via the network -based portal, based on the the audio message into a textmessage to be received by
identifier associated with the first user. the first user.
67. A non -transitory computer readable medium as recited 69. A non -transitory computer readable medium as recited
in claim 65, wherein said computer readable medium com 5
in claim 46 ,
prises computer program code for enabling the second user
to select a predetermined message to send to the first user via wherein the first message is a text message, and
at least one of the plurality ofmodes of communication , the wherein said computer readable medium comprises com
predetermined message being selected from a set of prede puter program code for transforming the first message
termined messages provided to the second user. 10 into an audio message so that the first message is
68. A non -transitory computer readable medium as recited received by the second user in an audio manner via a
in claim 67 , speaker if the first user is not blocked from using the
wherein the first message is a text message, selected mode of communication to communicate with
wherein said computer readable medium comprises com the second user .
puter program code for transforming the firstmessage
into an audio message so that the first message is 15 70. A non -transitory computer readable medium as recited
received by the second user in an audio manner via a in claim 69 , wherein said computer readable medium com
speaker if the first user is not blocked from using the prises computer program code for receiving an audio mes
selected mode of communication to communicate with sage from the second user to the first user, and transforming
the second user, and the audio message into a textmessage to be received by the
wherein said computer readable medium comprises com 20 first user.
puter program code for receiving an audio message
EXHIBIT C
US010142810B2
(12) United States Patent (10) Patent No.: US 10 , 142,810 B2

Cheung et al. (45) Date of Patent: *Nov . 27 , 2018
(54 ) METHOD AND APPARATUS TO MANAGE
DIFFERENT OPTIONS OF
(58 ) Field of Classification Search
CPC ... ........... . HO4W 76 / 02; H04W 88 / 184 ; H04M
COMMUNICATION USING ONE USER 1 /72552; H04M 3 /42059
IDENTIFIER BASED ON INTERNET (Continued )
PROTOCOL
(56 ) References Cited
(71 ) Applicant: Ip Venture, Inc., Los Altos, CA (US) U . S. PATENT DOCUMENTS
(72 ) Inventors: Kwok Wai Cheung , Tai Po (HK ); 5 ,425 ,516 A 6 / 1995 Daines
Peter P . Tong , Mountain View , CA 5 ,548 ,636 A 8/ 1996 Bannister et al .
(US); C . Douglass Thomas , Saratoga , (Continued )
CA (US )
(73) Assignee: IpVenture, Inc., San Jose , CA (US ) FOREIGN PATENT DOCUMENTS
CN 1453981 11 /2003
( * ) Notice : Subject to any disclaimer, the term of this WO WO 01/45343 A2 6 / 2001
patent is extended or adjusted under 35
U . S .C . 154 (b ) by 0 days . OTHER PUBLICATIONS
This patent is subject to a terminal dis Notice of Allowance for U .S . Appl. No. 12/798 ,995 dated Sep . 29 ,
claimer . 2011.
(21) Appl. No.: 15/469,440 ( Continued )
( 22 ) Filed : Mar. 24 , 2017 Primary Examiner — Kiet M Doan
(65) Prior Publication Data (57 ) ABSTRACT
Computer- implemented systems and methods to manage
US 2017 /0201872 A1 Jul. 13, 2017 electronic communications are disclosed . In one embodi
ment, an apparatus, using at least a network -based portal
Related U .S . Application Data based on Internet protocol, receives a message with a user
identifier from a person ' s wireless device ; receives a com
(63 ) Continuation of application No. 14 / 922, 344, filed on munication option from the person based on options pro
Oct. 26 , 2015 , now Pat. No . 9 ,736 ,664 , which is a vided to the person , with all the options using the user
identifier, in view of the portal being based on the Internet
( Continued ) protocol; permits the user to block the person from accessing
(51 ) Int. Ci. the user ; enables the message to be received by the user if
H04M 3 /42 (2006 . 01) the person is not blocked by the user ; and determines user
H04W 4 / 14 ( 2009.01 ) availability to receive the message . In the embodiment, the
apparatus requires user contact information for the user to
(Continued ) receive messages , with the user contact information not
(52) U . S . CI. provided to the person , and with the user contact information
CPC ............. H04W 4 / 14 (2013 .01); G06Q 10/ 10 being distinct from the user identifier.
(2013.01); H04M 3/436 ( 2013 .01);
(Continued ) 20 Claims, 9 Drawing Sheets
( START
202
INCOMING
VOICE NO
CALL
YES
204
200 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE 212
CALL
AUDIO 214
MESSAGE
ENGAGE IN
AUDIO MESSAGE
208
TO CALLEA
210
CALL
ENDED 216
TEXT
MESSAGE 218
YES YES
OBTAIN AND SEND
NO TEXT MESSAGE
TO CALLER
220
VOICE 222
MAIL YES
DIRECT TO
NO VOICE MAIL
DROP THE INCOMING . 224
VOICE CALL
END
US 10 ,Page
142,2810 B2
Related U . S . Application Data 7 , 346 ,630 B2 *

7 ,376 ,434 B2
3 /2008 Eichstaedt .............. GO6F 9 /542
5 /2008 Thomas et al.
continuation of application No. 14 /272 ,632 , filed on 7 ,403, 972 B1 7 /2008 Lau et al.
May 8 , 2014 , now Pat. No . 9, 204 , 268 , which is a 7 ,729 ,688 B2 6 /2010 Cheung et al.
continuation of application No. 12 / 798, 995 , filed on 7 ,792, 552 B2 9 /2010 Thomas et al.
Apr. 14 , 2010 , now Pat. No . 8 ,744 ,407, which is a 7 ,890 , 128 B1 2 / 2011Thomas et al.
8 , 112, 104 B1 2 /2012 Thomas et al.
continuation of application No. 11 /452 , 115 , filed on 8 , 280 ,419 B1 10 /2012 Thomas et al.
Jun . 12 , 2006 , now Pat. No. 7 ,759 ,688 , which is a 8 ,391 ,459 B2 3 /2013 Jackson et al.
continuation -in -part of application No. 11 /006 ,343 , 8 ,429, 231 B2 4 /2013 Wu et al.
filed on Dec . 7 , 2004 , now Pat. No. 7 , 116 , 976 . 8 ,737 ,978 B1 5 /2014 Thomas et al.
9 , 204, 268 B2 12 /2015 Cheung et al.
(60 ) Provisional application No. 60 /527 , 565, filed on Dec . 2001/ 0011014 AL 8 /2001 Higuchi et al.
8 , 2003, provisional application No. 60 /689,686 , filed 2001/0014611 Al 8 /2001 Dufort
2001/0028709 Al 10 /2001 Makela et al.
on Jun . 10 , 2005 . 2001/0031633 A1 10 /2001 Tuomela et al.
2002 /0067806 A1 6 /2002 Rodriguez et al.
(51 ) Int. Cl. 2002 /0073207 A1 6 /2002 Widger et al.
G060 10 / 10 ( 2012 .01) 2002 /0094067 Al 7 /2002 August
H04M 3 /436 ( 2006 .01) 2002 /0142756 AL 10 /2002 Rutledge et al.
H04W 4 / 16 2003/ 0039339 A1 2 /2003 Luehrig et al.
(2009.01) 2003 /0041048 A1 2 / 2003 Balasuriya
H04W 4 / 12 ( 2009.01) 2003/0065779 A1 4 /2003 Malik
(52 ) U .S . CI. 2003/0103600 A1 6 / 2003 Potter
CPC .. . .... H04M 3 /4365 (2013 .01) ; H04W 4 / 12 2003/0105854 A1 6 / 2003 Thorsteinsson et al.
2003 /0112948 A1 6 /2003 Brown et al .
(2013 .01) ; H04W 4 / 16 ( 2013.01); H04M 2003/0129968 A1 7 / 2003 Earl
2201/60 ( 2013 .01); H04M 2203 /2011 2003 /0191676 Al 10 / 2003 Templeton
(2013.01); H04M 2203/651 (2013.01); H04M 2003/0191814 Al 10 / 2003 Tran
2003/0232629 A1 12 /2003 Jang et al.
2207 /18 (2013 .01 ) 2004 /0024882 Al 2 / 2004 Austin et al.
(58) Field of Classification Search 2004 /0072585 A1 4 /2004 Le et al.
USPC ... 455 /466 , 456 .1, 414 . 1, 557 , 426 .1 , 435 .1 , 2004 /0122979 Al 6 / 2004 Kirkland
455 /424 , 455 , 423 , 39 , 417 ; 379 /88 .04 , 2004 /0203919 A 10 /2004 Ross et al.
379 /268, 88 .25, 210 .01; 340/ 7 .22, 7 .42 ; 2004/ 0240650 Al 12 / 2004 Bear et al.
709/ 206 , 247; 704 /235 2004/ 0248596 A1 * 12 / 2004 Panchal H04W 4 /06
See application file for complete search history . 455 /466
2005/ 0020288 Al 1/ 2005 Davis
2005 /0027385 Al 2 / 2005 Yuch
( 56 ) References Cited 2005 /0037785
2005 /0038690
A1
A1
2 / 2005 Chen
2/ 2005 Hayes -Roth
U .S . PATENT DOCUMENTS 2005 /0071253 A1 3/2005 Yang
2005 /0107130 A1 5 /2005 Peterson , II
5 ,610 ,970 A * 3/1997 Fuller .... ............ H04M 1 /006 2005 /0136955 Al 6 /2005 Mumick et al.
379 /207.03 2005 /0191994 Al 9 /2005 May et al.
5 ,752, 191 A * 5/1998 Fuller ........ ... H04M 1 /006 2005/0192061 AL 9/2005 May et al.
455 /445 2005 /0273327 All 12 / 2005 Krishnan
5 ,758 ,079 A 5 / 1998 Ludwig et al. 2006 /0003803 A11 /2006 Thomas et al.
5 ,828, 731 A 10 / 1998 Szlam et al. 2006 /0075038 A14 /2006 Mason et al .
5 ,930 ,700 A * 7 / 1999 Pepper .................. H04M 1/663 2006 /0239419 A1 10 /2006 Joseph et al .
379 /211.02 2006 /0259565 AL 11/2006 Cheung et al.
5 ,970, 388 A 10 / 1999 Will 2006 / 0276210 A112/ 2006 Thomas et al.
6 , 119 , 022 A 9 / 2000 Osborn et al. 2006 /0288099 Al 12 /2006 Jefferson et al.
6 ,327,628 B1 12 / 2001 Anuff et al. 2007/ 0005368 A11 / 2007 Chutorash et al.
6 ,359,982 B1 3 / 2002 Szlam et al. 2007 /0047522 AL 3 /2007 Jefferson et al.
6 ,463,462 B1 10 / 2002 Smith et al. 2007/ 0238474 Al 10 / 2007 Ballas
6 ,577 ,859 B1 6 /2003 Zahavi et al . 2008 /0261636 A1 10 /2008 Lau et al.
6 ,636 , 888 B1 10 / 2003 Bookspan et al. 2010 /0114958 Al 5 /2010 Korenshtein
6 ,665 ,534 B112 /2003 Conklin et al. 2010 /0205272 Al 8 /2010 Cheung et al.
6 ,788 ,766 B2 9 / 2004 Logan 2011 /0151582 Al * 6 / 2011 Basile . . .. .. . .. .. · GOIN 33 / 585
6 ,801,793 B110 / 2004 Aarnio et al . 436 /501
6 ,816 ,578 B1 11/ 2004 Kredo et al. 2011 /0151852 A1 6 / 2011 Olincy et al .
6 ,819 ,757 B1 11/ 2004 Cook et al. 2014 /0242956 Al 8 / 2014 Cheung et al.
6 , 819 ,945 B1 11/2004 Chow et al. 2014 /0256293 Al 9 / 2014 Thomas et al.
6 ,978 ,136 B2 12/ 2005 Jenniges et al. 2016 /0044474 A1 2 /2016 Cheung et al.
7 ,010 ,288 B2 3 / 2006 Brown et al. 2018 /0014169 AL 1 /2018 Cheung et al.
7 ,010 ,332 B1 3 / 2006 Irvin et al.
7 ,027 ,842 B2 4 / 2006 Zhang et al. OTHER PUBLICATIONS
7 ,043 , 261 B2 5 / 2006 Krishnan
7 ,072 ,452 B1 7 / 2006 Roberts et al. Notice of Allowance for U .S . Appl. No. 12/798 ,995 dated Jan . 3,
7 ,085 ,253 B2 8 / 2006 Yang
7 , 107 ,010 B2 9/ 2006 Heinonen et al. 2012.
7 , 110 , 963 B2 9/ 2006 Negreiro Notice of Allowance for U .S. Appl. No. 12 /798, 995, dated Oct. 3,
7 ,111 ,044 B2 9 / 2006 Lee 2012 .
7 , 116 , 976 B2 10 / 2006 Thomas et al . Notice of Allowance for U .S . Appl. No. 12 /798 ,995, dated Oct. 30 ,
7 , 188 ,073 B1 3/ 2007 Tam et al. 2013.
7 ,224, 775 B15 / 2007 Shaffer et al. Notice of Allowance for U . S . Appl. No. 12 /798 , 995 , dated Feb . 20 ,
7 ,317,706 B1 * 1/2008 Hao .................. H04W 72 / 1242 2014 .
370 /329 Office Action for U . S . Appl. No. 14 /272 ,632 , dated Jul. 27, 2015 .
US 10 ,Page
142,3810 B2
( 56 ) References Cited IMBOT, Press Release, “ IMBOT offers new Text 2 Voice Service
Text 2 Voice service enables wireless customers to send voice
OTHER PUBLICATIONS messages from 2 -Way devices” , Oct. 29 , 2001, pp . 1- 2 .
Internet Traveler, “ Welcome to the Inter.Net Communicator Tour!" ,
Notice of Allowance for U .S . Appl. No . 14 /272 ,632 , dated Sep . 18 , http ://www .inter.net/traveler/tour/communicator_messaging.php , down
2015 . loaded Oct. 14 , 2003, p . 1.
Office Action for U . S . Appl. No. 14 /922 ,344 , dated Apr. 27 , 2016 . J. Rosenberg , H . Schulzrinne, Internet Draft, “ SIP for Presence,"
Office Action for U .S . Appl. No. 14 /922 ,344 , dated Oct. 7, 2016 . http : //www .alternic .org/ drafts/ drafts -r -s/ draft- rosenberg = sip -pip -00 .
First Office Action for CN Patent Application No. 200680027964.9 , txt, Nov . 13 , 1998 , Bell Laboratories, Columbia , pp . 1 -31 .
Joseph , Anthony D . et al., " The Case for Services over Cascaded
dated Mar. 26 , 2010 ( 17 pages ). Networks” , EECS Department, CS Division , University of Califor
Second Office Action for CN Patent Application No. 200680027964. nia , Berkeley , http : //iceberg .cs .berkeley .edu /, International Confer
9 , dated Oct. 25 , 2010 ( 14 pages ). ence on Wireless and Mobile Multimedia 1998 , pp . 1 -9 .
Third Office Action for CN Patent Application No. 200680027964. MobileShop , " SMS — also know as text messaging” , http ://www .
9 , dated Apr. 8 , 2011 ( 11 pages ). mobileshop .org/howitworks.sms.htm , downloaded Oct. 14 , 2003,
Notice of Rejection for CN Patent Application No. 200680027964 . pp . 1 -2 .
9 , dated Jan . 6 , 2012 (11 pgs.). Schulzrinne , H . et al., RPID : Rich Presence Extensions to the
" Company Overview ” , http ://www .fastmobile .com /company _overview . Presence Information Data Format (PIDF), draft -ietf = simple - rpid
html, downloaded Nov. 5 , 2003, p . 1. 06 .txt, Jun . 2 , 2005 , http ://www1.ietf.org /mail -archive /web /simple/
“ Introducing the Tellme Voice Application Network ” , Tellme, http :// current/msg05398 .html, downloaded Nov. 15 , 2006 , pp . 1 - 35 .
www .tellme.com / products/, downloaded Oct. 2 , 2003 , p . 1 . Schulzrinne, H . et al., RPID : Rich Presence Extensions to the
“ iotum History," Lotum Corp ., http ://iotum . com /simplyrelevant/ Presence Information Data Format (PIDF ), draft -ietf-simple- rpid
2006 /04 /03 /iotum -history /, downloaded May 15 , 2006 , pp. 1 -4 . 10 .txt, Dec . 20 , 2005, pp . 1 -41.
"messaging ” , Vodafone Group , 2001 , http :www .vodafone .co .nz Schulzrinne, H . et al., RPID : Rich Presence Extensions to the
business/ 10 .2 .3 _messaging.jsp, downloaded Oct. 14 , 2003, pp . 1- 2 . Presence Information Data Format ( PIDF), draft-ietf- simple - rpid
“ Microsoft Windows Messenger: Go Beyond Text with Voice & 10 .txt, Dec . 4 , 2005 , pp . 1 - 35 .
Video Chats” , Dell Inc ., http ://www .dell.com /us /en / dhs /topics/ Sonim Technologies , Inc ., “ Integrated voice and text messanging
segtopic _ 002_ xp _ im .htm , downloaded Oct. 2 , 2003, pp . 1 -2 . over GPRS showcased jointly by Sonim , Symbian and Texas
“ Microsoft WindowsMessenger: Instantly Communicate with Fam Instruments” , Sonim Press Release, Dec . 2 , 2002, pp . 1 -2 .
ily and Friends Messenger” , Dell Inc., http ://www .dell.com /us/en / Symbian Ltd ., “ Symbian OS Version 7 .0 : Functional description ” ,
dhs/topics/segtopic _ 001_ xp _ im .htm , downloaded Oct. 2 , 2003 , pp . Revision 1.5 , Feb . 2003, pp . 1 -24 .
1 - 3. Symbian Ltd ., “ Symbian OS Version 7.0s: Functional description " ,
“ Our Solution ,” Lotum Corp ., http ://www .iotum .com / our _ solution . Revision 2 . 1, Jun . 2003, pp . 1 -29 .
php , downloaded May 15 , 2006 , pp . 1 -2 . Symbian Ltd ., “ Technology : Creating Symbian OS phones” , http ://
Short Message Service/Interactive Voice Response (SMS/IVR ), www .symbian .com /technology /create -symb -OS -phones.html, down
Lucent Technologies , 2003 , pp . 1 - 2 . loaded Nov. 5 , 2003, p . 1- 8.
“ text messaging” , Vodafone Group , 2001, Vodafone Services, Symbian Ltd ., “ Technology : Why is a different operating system
" All about text messaging " , http ://www .vodafone .co .nz/services/07 . needed ” , http ://www .symbian .com /technology /why -diff-os.html,down
a. 1 _two _way _ messaging. jsp ?hd = 4yourbusiness & . . . , downloaded loaded Nov. 5 , 2003, pp . 1 -5 .
Oct. 14 , 2003, pp . 1 - 2 . Verizon Wireless, “ TXTmessaging” ,http ://www .vtext.com /customer _
“Webring relevance to communications,” Cnet News, Ina Fried , site/jsp /messaging _ lo .jsp , downloaded Oct. 2 , 2003 , p . 1 .
Jul. 21, 2005, pp . 1 -2 . W3C , Voice Extensible Markup Language (VoiceXML) Version
Appenzeller, et al., “ The Mobile People Architecture ” , Technical 2 .0 , W3C , www .w3.org , Feb . 20 , 2003.
Report: CSL - TR - 00000 , Computer Systems Laboratory , Depart Yahoo !Messenger , “ Yahoo !Messenger Talk for Free !” , http ://
ments of Electrical Engineering and Computer Science , Stanford messenger.yahoo .com /messenger/help /voicechat.html, downloaded
University, Jan . 1999 , pp . 1 - 13 . Oct. 2 , 2003 , pp . 1 - 2 .
BlackBerry , “ Voice and SMS” , http ://www .blackberry .com /products / Notice of Allowance for U .S. Appl. No. 12/798,995 , dated May 9 ,
service /voices _ sms.shtml?DCPID = hmsvoice downloaded Oct. 2 , 2012 .
2003, p . 1. Notice of Allowance for U . S . Appl. No. 12 / 798 ,995 , dated Jul. 16 ,
Calsyn ,Martin and Desseault , Lisa , “ Presence Information Protocol 2013 .
Requirements," Internet Draft, Feb . 9 , 1998 , pp . 1 -27. Notice of Allowance for U .S . Appl. No. 14 /922,344, dated Feb . 14 ,
Emergin Inc ., “ Emergin WirelessOffice 5 . 0 " , http ://www . emergin . 2017 .
com / ? source = overture , downloaded Oct. 2 , 2003 , p . 1 . Notice of Allowance for U .S . Appl. No. 14 /922,344 , dated Mar. 6 ,
fastmobile Inc ., “ Dialog GSM launches Push ‘n ’ Talk walkie talkie 2017 .
service Push to Talk over Cellular Now in Sri Lanka Dialog GSM Notice of Allowance for U . S . Appl. No. 15 / 704 , 181, dated Apr. 27 ,
Pioneers Latest GSM Advancement” , Press Release, Dec . 1, 2004 , 2018 .
pp . 1 - 2 .
fastmobile, " fastmobile ' s fastchatTM Instant Communications Appli
Notice
2018 .
of Allowance for U .S. Appl.No. 15 /704,181, dated Aug. 22,
cation is Coming to Thousands of Mobile Phone Retail Stores
Nationwide” , fastmobile Press Release, Sep . 15 , 2003, pp . 1 -3 . * cited by examiner
U.S.Patent
atent Nov . 27, 2018 Sheet 1 of 9 US 10 , 142 ,810 B2
ICM Default
1 Mobile phone Voice mail
Ollice phone Voice mail
Home phone Voice mail
Mobile SMS/pager from mobile phone or PDA Email
Home/office SMS (to office home PC ) Email
Mobile Online chat ( to mobile phone or PDA ) Voice mail
Home Online chat (NetMeeting , AOL , ICQ etc . ) Voice mail
Voice mail with instant notification to mobile devices of the user
Oflice fax
Home far Reject
Mobile Email ( Blackberry etc .) Email
Email Reject
14 User defined
com 42000
!
????

ContactClass 2 Relatives and friends
ContactClass3 Boss and VIP
ContactClassy Secretary
ContactClass 10 User defined
bo o ????? ba I
atent Nov . 27, 2018 Sheet 2 of 9 US 10 ,142 ,810 B2
UrgClassi Life threatening - interrupt at any one and occasion

UrgClass2 Urgent confirmed meeting reminder - interruption
allowed
UrgClass3 Urgentmatter requiring immediate allention
UrgClass4 Importantmatter requiring quick attention
UrgClassi Regular work related matter
UrgClassó Casual contact
UrgClass7 Cold calls from unknown person
UrgClass User defined
} Soos 66 podo
MyBusy Statel Inportant ineeting

MyBusyState2 Ordinary meeting
MyBusyState4 Sleeping
MyBusyState5 Resting
MyBusy State User defined
dopod 1
ContactClass UrgClass MyBusyState ICM allowed

ContactClass2 UrgClass 1- 3 All
UrgClass 4 - 6 MyBusyStatel All
MyBusy State2 - 3
MyBusyState4 -5
UrgClass 7- 8 ICM 13
1 ?. november
- 100
--
106
--
--- -----
102
NETWORK
108
104
104
106 on 102
FIG . 6
START 200
202
INCOMING
VOICE NO
CALL
YES
204
206 ANSWER
YES
ANSWER THE NO
INCOMING VOICE 212
CALL
AUDIO
MESSAGE 214
YES
ENGAGE IN
NO AUDIO MESSAGE
TO CALLER
208 210
CALL
ENDED 216
NO TEXT 218
MESSAGE
YES YES
OBTAIN AND SEND
NO
TEXTMESSAGE
TO CALLER
220
VOICE
MAIL YES
DIRECT TO
NO VOICE MAIL
DROP THE INCOMING
VOICE CALL 224
FIG . 7
END
atent Nov . 27, 2018 Sheet 5 of 9 US 10 ,142,810 B2
300
START
START 300
PRESENT ONE OR MORE PREDETERMINED I 304
AUDIO MESSAGES
306
PREDET.
AUDIO
MESSAGE NO
SELECTED 310
YES
CUSTOM
AUDIO
308 MESSAGE
REQUESTED NO
PLAY SELECTED AUDIO
MESSAGE FOR CALLER
YES
RECORD CUSTOM AUDIO 312

MESSAGE
PLAY CUSTOM AUDIO N314

MESSAGE FOR CALLER
END
FIG . 8
400
(START
START ) 400
DROP THE INCOMING VOICE CALL 1 402
DISPLAY ONE OR MORE PREDETERMINED 404

TEXT MESSAGES
406
PREDET.
TEXT
MESSAGE NO
SELECTED r410
YES CUSTOM
TEXT
408
MESSAGE
REQUESTED / NO
TRANSMIT SELECTED
TEXT MESSAGE
TO CALLER YES
ENTER CUSTOM
412
TEXT MESSAGE
TRANSMIT CUSTOM
TEXT MESSAGE 414
TO CALLER
END
FIG . 9
START
- 500
202
INCOMING
VOICE
CALL
YES
204
206 ANSWER
YES ?
ANSWER THE NO
INCOMING VOICE
CALL 502
HEAD -SET 214
ACTIVE YES
ENGAGE IN ?
AUDIO MESSAGE
TO CALLER
208 m 210
CALL
ENDED 504
NO
DISPLAY 218
YES PRESENT / YES
OBTAIN AND SEND
NO TEXT MESSAGE
TO CALLER
n 220
VOICE 222
MAIL / YES
DIRECT TO
NO VOICE MAIL
DROP THE INCOMING
FIG . 10 VOICE CALL 224
( END
U.S.Paten
600
START
602 ^
INCOMING
TEXT
NO
MESSAGE
YES
604 ^
AUDIO
OR TEXT
PRESENTATION 610
TEXT
TEXT MESSAGE
AUDIO
606 ^ CONVERT THE INCOMING TEXT

END
FIG . 11
U .S. Patent Nov.21,2018

START
- 700
702 ^
REPLY
MESSAGE
NO TO BE
SENT 714
YES DISPLAY TEXT

ENTRY SCREEN
704
AUDIO
OR TEXT
MESSAGE TEXT
TO BE MESSAGE
TEXT
FORMED ENTERED
706 716 YES

AUDIO
708
AUDIO
MESSAGE
RECORDED NO
710
YES
TO A TEXT MESSAGE
TRANSMIT THE TEXT MESSAGE V 712

OVER A WIRELESS NETWORK
END FIG . 12
US 10 ,142 ,810 B2
METHOD AND APPARATUS TO MANAGE network - based portal based on Internet protocol, could pro
DIFFERENT OPTIONS OF vide a number of communication options to a first user, with
COMMUNICATION USING ONE USER all the options using an identifier associated with a second
IDENTIFIER BASED ON INTERNET user for the second user to receive messages via an elec
PROTOCOL 5 tronic device associated with the second user, the options
including text messaging, voice communication , multimedia
CROSS -REFERENCE TO RELATED messaging, and group messaging ; could receive an indica
APPLICATIONS tion from the first user via an electronic device associated
with the first user, indicating the selection of a communi
This application is a continuation of U . S . patent applica - 10 cation option for a message for the second user ; could permit
tion Ser. No . 14 /922,344 , filed Oct. 26 , 2015 , now U . S . Pat the second user to block the first user from accessing the
No. 9,736 ,664, which is hereby incorporated herein by second user ; and could determine availability of the second
reference , which application is a continuation of U . S . patent user to receive the message . In the embodiment, the appa
application Ser. No . 14 /272 .632, filed May 8 , 2014 . now ratus could require contact information associated with the
U .S . Pat. No . 9 ,204 ,268, and entitled “ SYSTEMS AND og
15 second user to allow the second user to receive messages via
PROCESSES TO MANAGE MULTIPLE MODES OF the network -based portal, with the contact information asso
COMMUNICATION ,” which is hereby incorporated herein ciated with the second user not provided to the first user via
by reference , which application is a continuation of U . S . the electronic device associated with the first user, even
patent application Ser. No . 12 /798, 995 , filed Apr. 14 , 2010 , when the message is received by the second user via the
now U . S . Pat.No. 8 ,744, 407, and entitled “ SYSTEMS AND 20 electronic device associated with the second user, and with
PROCESSES TO MANAGE MULTIPLE MODES OF the contact information associated with the second user
COMMUNICATION ,” which is hereby incorporated herein being distinct from the identifier associated with the second
by reference , which application is a continuation of U . S . user.
patent application Ser . No. 11 /452, 115 , filed Jun . 12 , 2006 , A person tries to electronically convey a message to the
now U . S . Pat.No. 7 ,729 .688, and entitled “ SYSTEMS AND 25 user. In one embodiment, the status of the user is identified ;
PROCESSES TO MANAGE MULTIPLE MODES OF the identity of the person is identified ; the urgency of the
COMMUNICATION ” , which is hereby incorporated herein message is identified ; the access priority of the person is
by reference, which application is a continuation - in -part determined based on the person ' s identity ; and a process is
application of U . S . patent application Ser. No. 11/006 , 343, set to manage the message using one or more rules , and in
filed Dec. 7 , 2004 , now U . S . Pat. No. 7 , 116 , 976 , and entitled 30 view of the status of the user, the access priority of the
“ ADAPTABLE COMMUNICATION TECHNIQUES FOR person and the urgency of the message .
ELECTRONIC DEVICES .” which is hereby incorporated Based on different embodiments, the status of the user
herein by reference, which claims priority to U . S . Provi depends on the current activity or location of the user, or the
sional Patent Application No. 60 /527, 565, filed Dec. 8 . current time. The status of the user can also be defined by the
2003 , entitled “ ADAPTABLE COMMUNICATION TECH - 35 user. Similarly , the access priority of the person can be
NIQUES FOR ELECTRONIC DEVICES ,” and which is defined by the user, or is set depending on the user 's reaction
hereby incorporated herein by reference . at anplication towards a prior message from the person . Also , the urgency
This application , by way of U . S . patent application Ser. of the message is set by the person .
No . 11 /452. 115 . also claims priority to U . S . Provisional The process can depend on themode of communication of
Patent Application No . 60/689,686 , filed Jun . 10 , 2005 , 40 the message . For example , the mode of communication can
entitled “ SYSTEMS AND PROCESSES TO MANAGE include a mobile phone , an office phone, a home phone, a
MULTIPLE MODES OF COMMUNICATION .” and which mobile SMS, a pager from a mobile phone or PDA , a
is hereby incorporated herein by reference . home/office SMS, mobile online chat, home online chat, a
voice mail with /without instant notification , an office fax , a
BACKGROUND OF THE INVENTION 45 home fax, a mobile email , and an email.
In one embodiment, the user receives the message
For many years , other than mails from post offices , we through a handheld device , such as a cellular phone. In
typically only received information from afar through tele - another embodiment, themessage is electronically conveyed
phones . However , in the past few years , ways that others can based on Internet protocol through a website .
send us information have increased significantly . Just to list 50 In one embodiment, though the process allows the user to
a few different modes of communication , we can be reached receive the message , the person is not aware of the contact
from standard desk phones, fax , cell phones , electronic information of the user. For example, the person is not aware
mails, and instant messages . In addition , we can have more of the phone number of the cellular phone that the user used
than one phone number and multiple electronic mail to talk to the person . This prevents the person from directly
addresses . There are people we like to communicate with , 55 accessing the user without going through an intermediate
and there are those we prefer to avoid . Managing informa control, such as a website . Similarly , the user does not have
tion from all such different modes can be quite time con- to be aware of the contact information of the person .
suming . In another embodiment, the defined access priority of the
It should be apparent from the foregoing that there is still person is stored at a website , allowing the website to access
a need to help manage the numerous modes of communi - 60 such information without asking for the user 's permission .
cation . In one embodiment, the defined access priority is stored in
a private database under the user 's control.
SUMMARY OF THE INVENTION In one embodiment, textmessages could be received in an
audio manner, and audio messages could be sent as text
Different embodiments of a computer- implemented sys - 65 messages.
tem and method to manage the communication of a user are Other aspects and advantages of the present invention will
disclosed . In one embodiment, an apparatus , using at least a become apparent from the following detailed description ,
US 10 , 142 ,810 B2
which , when taken in conjunction with the accompanying tion from the differentmodes. The user only has to remem
drawings, illustrates by way of example the principles of the
ber one specific address from one mode of communication .
invention . Through that address , the user can receive communications
from all modes of communication , independent ofwhere the
BRIEF DESCRIPTION OF THE DRAWINGS 5 user is, or the type of hardware the user has. This allows the
user to efficiently maintain his communication from the
FIG . 1 shows a number of intelligent communication numerous modes even when he is traveling. For example ,
modes according to one embodiment of the invention . the user does not have to change phones ( and the phone
FIG . 2 shows a number of contact classes according to numbers ) when he moves from areas covering 3G to areas
one embodiment of the invention . 10 that do not .
FIG . 3 shows a number of urgency classes according to A number of embodiments depend on the differentmodes
FIG . 4 shows a number of statuses of a user according to platform . A communication gateway or a portal is formed
FIG . 5 shows one embodiment of an example of an 15 sources through differentmodes. This , in turn , could reduce
Access Priority Database according to one embodiment of the numerous addresses the user has to remember, to one
the invention . address. For example, an e -mail address for the user can
embodiment of the invention . addresses from different communication modes. The access
FIG . 7 is a flow diagram of a personal call response 20 identifier can become the user ' s digital identity . In one
process according to one embodiment of the invention . embodiment, the user ' s other types of identification , such as
FIG . 8 is a flow diagram of an audio message response the user ' s driver licenser number, can be the user' s access
process according to one embodiment of the invention. identifier.
process according to one embodiment of the invention . 25 based on the web . Based on the portal, the user can securely
FIG . 10 is a flow diagram of an automated call response determine who can reach him at what conditions . This can
process according to one embodiment of the invention. be done based on a status indicator. As an example , this
process according to one embodiment of the invention . the access priorities of the person trying to reach the user (or
FIG . 12 is a flow diagram of a message presentation 30 the relationship or the lack of relationship between the user
process according to one embodiment of the invention . and the person ), and /or the urgency of the message from the
Same numerals in FIGS . 1- 12 are assigned to similar person . The status of the user can be dynamically deter
elements in all the figures. Embodiments of the invention are mined , based on the current condition (s) of the user . The
discussed below with reference to FIGS. 1 - 12 . However, portal can allow the user and the person to select different
those skilled in the art will readily appreciate that the 35 options, which can be modified as desired . For example , the
detailed description given herein with respect to these fig . relationship can be preset by the user and stored in a
ures is for explanatory purposes as the invention extends database , while the urgency of the message can be set by the
beyond these limited embodiments. person .
Thus, in one embodiment, the portal can be used to
DETAILED DESCRIPTION OF THE 40 control the selection and setting of different intelligent
INVENTION communication modes for the user. These intelligent com
munication modes allow priorities of various kinds of com
One embodiment of the invention can automatically munication options to be set by the user. The portal allows
remove unwanted communications. Certain communica - worldwide access to the user, and can dynamically deter
tions are relatively easy to determine to be unwanted , such 45 mine , for example , whether a call initiated at different time
as marketing cold calls and wrong number calls. Other by different callers should be accepted by the user in
communicationsmay be more difficult. They can depend not real-time or handled by other mechanisms. From this infor
conditions or status of the receiver (a user ) of the commu- example , into different degrees of undesirability . Some
nication . The status can be related to the user ' s current 50 requests can be automatically blocked from the user. Others
activity and /or location . For example, when the user is on a can be diverted and handled by other mechanism , such as
train going to work , the user probably does not mind diverting a phone call to an email or voice mail .
chatting with his grandchild . However, if the user is having In one embodiment, the portal or gateway also includes a
his yearly review meeting with his boss , the user probably database to keep track of the user 's different contacts or
would prefer to avoid the call from his grandchild , unless it 55 acquaintances , and the access priorities of each contact. The
is an emergency . Based on the embodiment, communica - user can modify information in the database , such as assign
tions from sources the user wants to postpone receiving can ing and/ or changing the priorities of the contacts . Based on
be automatically diverted . the information ( or lack of information ) in the database of
In one embodiment, the user can get appropriate notifi - the contact trying to access the user, and based on the status
cation on the source of the incoming communication 60 of the user, the gateway can automatically select an intelli
request. The attributes of the notification can depend on the gent mode of communication for the user. This selection can
urgency of the communication and/or the status of the user. be done dynamically.
The usermay receive information from differentmodes of In one embodiment, the portal can dynamically change
communication . For example , the user can have mobile the access priorities of a caller trying to reach the user . For
phones, fixed lines at home or office , emails, SMS, and 65 example , previously the caller is ofhigh priority to the user,
faxes , with their different numbers and/ or addresses . One and the user has set her access priorities accordingly . Lately ,
embodiment can help the user efficiently manage informa every time the caller trying to reach the user , the request was
US 10 ,142 ,810 B2
denied . After a preset number of rejections, the portal can default because it may not be necessary to default . For
automatically send a message to the user, asking the user if example, under ICM 8 , the incoming message goes directly
the user would like to lower the access priority of the caller. to voice mail with instant notification to mobile devices of
If the response is affirmative, the caller ' s priority is auto - the user. The incoming message can usually go to voice
matically reduced . 5 mail. There is no need to default.
In another embodiment, the user does not have to set As a receiver of communication , the user can define a
priorities of each contact. The system monitors every call, number of contact classes , as shown in FIG . 2 . The user can
and provides the contact's identity to the user. Based on the set up a number of urgency classes, as shown in FIG . 3 . The
user' s reaction to the call ( e .g . accepting or rejecting it), the user can define a number of status, as shown in FIG . 4 . Then ,
system automatically sets the contact 's priorities. In one 10 based on tables in FIGS. 1 -4 , the user can set up an Access
embodiment, the system can then query the user for approval Priority Database for different ContactClasses, as shown in
on the setting, and allow the user to adjust it as necessary . In FIG . 5 .
another embodiment, the system can continue to modify the As another example , the user can categorize the following
caller ' s priorities based on the user 's reaction to the caller 's contacts into the corresponding ContactClasses :
subsequent calls. 15
believes to be sensitive local in a different database . Such Alice (Wife ) ContactClass 1
information can be stored securely under the user 's direct Peter ( Close Friend ) ContactClass 2
control. The portal can retrieve information from the differ Colina (Close Friend 's wife ) ContactClass 2
ent database when required . In another embodiment , the user 20
can restrict or limit such retrieval process . Peter wants to make a mobile phone call to the user. In one
Additional confidentiality can be provided . In one embodiment, Peter calls a portal. As an example, the portal
embodiment, using phone calls as an example , the user can can be the user's ISP. The portal first verifies the caller' s
be aware of the identity of the caller even without being identity to be Peter. This can be done , for example , by a
informed of the number of the caller. Similarly, the caller can 25 public key challenge based on Peter having a public key
reach the user without being aware of the number of the digital certificate . In another example , Peter is also a regis
phone the user is using to receive the call . The user can keep tered user of the portal. Then , Peter 's identity can be more
his location and / or status confidential but still can receive readily identified or verified .
the communication . This can be useful because there are In one embodiment, after verification , a virtual address /
situations , for example , when the user does not want to 30 number for the communication session is created allowing
disclose his contact information but the user needs to receive Peter to reach the user , which can be by phone . The user' s
services provided by the caller. phone number does not have to be disclosed to Peter.
One approach to maintain such confidentiality while Similarly , Peter 's mobile phone number does not have to be
maintaining real- time communication is based on a system disclosed to the user. The portal can assure the user that the
that digitally identifies the identities of the caller and the 35 person calling is Peter based on an identification verification
receiver .Note that the term caller is used in general. It is not process , such as ones described above .
just limited to phone calls , but they can be any person or In establishing contact , the portal can access the user' s
entity requesting to communicate with the user, such as database and determine that Peter belongs to ContactClass2 .
trying to send a message to the user. As a separate note , the The database can , for example, be in the portal.
caller can also be a user of different embodiments of the 40 In another embodiment, the database is in a personal
invention . communication device of the user. The portal accesses the
After determining the identities , the system can establish personal communication device to determine Peter 's Con
connections between the caller and the user in real time. tactClass .
Though contacts are established , the system only needs to Based on the ContactClass information , the status of the
ensure the identities of the caller and the user to each other. 45 user and Peter ' s urgency setting , the user may receive
However, the system does not have to disclose the phone Peter 's call directly . As another example , Peter may be asked
numbers , electronic addresses, physical locations and /or to leave a voice mail to the user , while the user is notified by
other attributes of the caller and the user to each other. In one a mobile short message regarding an incoming call from
embodiment, real time implies that the time required for the Peter.
identification is similar to the typical time required to set up , 50 As additional examples, in one embodiment, location
for example , a telephone call. The system can be a portal information of the user could be determined based on GPS
based on the web . information from , for example , the user ' s cell phone.
In one embodiment, a portal also holds the user 's elec In one embodiment, the user receives messages through a
tronic calendar. The calendar can be programmable , with handheld device , such as a phone, and the phone has a
entries set by the user . The portal can automatically and 55 switch . The switch can be a physical button or a software
securely set appointments for the user since the portal knows setting , such as a pull-down menu . The user could set his
the identity of the caller, and the status and schedule of the status dynamically by changing the physical or logical
user . For example , the appointment can be for a conference position of the switch . For example , one position can
call. indicate that the user is very busy , and should only be
To illustrate , in one embodiment, a portal provides a 60 interrupted by an urgent message from the user 's closest
number of intelligent communication modes (ICM ) for the contacts, such as his wife or parents . Another position can
user to select as shown in FIG . 1 . There are three columns indicate that the user 's status allows the user to receive any
in the table. If the communication mode selected in the messages from anyone .
second column does not work , the portal automatically As explained above , based on an embodiment, a message
defaults to the corresponding approach in the third column. 65 is electronically conveyed by a central network server, such
For example , under ICM 1 , if the mobile phone is busy, as a web server based on Internet protocol . A portal or
default to voice mail. Some of the selections do not have any gateway approach could provide general Internet access to
US 10 , 142 ,810 B2
one or more embodiments of the communication manage- described above can use the messaging approaches (e. g.,
ment systems so that users can configure the system behav - audio or textual messages ) described in U . S . patent appli
ior they desire . The portal or gateway can then facilitate cation Ser . No. 11 / 006 , 343 .
download of a database or update thereto to a communica Different embodiments of the invention pertain to
tion device , such as a phone. 5 improved approaches for users of electronic devices to
Also , as explained above, based on an embodiment, a user audio communicate with one another . The electronic devices have
could efficiently maintain his communication , and does not and /or textual output capabilities . The improved
even have to change phones when he moves from areas ways depending approaches can enable users to communicate in different
covering 3G to areas that do not. These phones could be 10 prior history, timeonordevice other
configuration , user preferences ,
criteria . In one embodiment, the
based on different communication mechanisms, such as communication between users is achieved by short audio or
GSM , CDMA , 3G and 4G systems. Also as explained above, textual messages .
the user could keep information in local databases, such as The electronic device can be any computing device hav
in such a phone. For example , the intelligent communication ing communication capabilities . Such computing devices
modes shown in FIG . 1 for the user to select are inin the
lect are the phone
phone.. 1515 can
can be refer to as communication devices. Examples of
be referred
The user could define the contact classes, such as the ones electronic devices include personal computers , personal
shown in FIG . 2 ; set up the urgency classes, such as the ones digital assistants, pagers or mobile telephones.
shown in FIG . 3 ; define the statuses , such as the ones shown Embodiments of the invention are discussed below with
in FIG . 4 ; set up the Access Priority Database , such as the reference to FIGS. 6 - 12 . However, those skilled in the art
one shown in FIG . 5 ; and categorize a number of the user 's 20 will readily appreciate that the detailed description given
contacts into the corresponding ContactClasses, all in the herein with respect to these figures is for explanatory
phone. When a caller places a call to the phone, based on purposes as the invention extends beyond these limited
information previously set in the phone and based on the embodiments .
urgency class selected by the caller, the phone could auto FIG . 6 is a communication system 100 according to one
matically manage the communication . Note that the phone 25 embodiment of the invention . The communication system
does not have to be a cellular phone . In one embodiment, the 100 can support different communication devices , including
phone is a desk top phone. mobile telephones 102 , computers 104 (e . g ., personal com
Again as explained above , the person or the caller trying puters ) and/ or wireless personal digital assistants (PDAs )
to contact the user could select different options. For 106 . Users of the communication devices 102 - 106 can
example , the urgency of themessage can be set by the caller . 30 communicate with like or different communication devices.
This selection is typically in the call setup phase . In one Each communication device 102 - 106 offers one or both of
embodiment, the caller has pre -selected the urgency class audio or textual communication capabilities. These commu
before making the call . In another embodiment, if the caller nication devices 102 - 106 can inter - communicate with one
has not selected the urgency class, the system could prompt another through a network 108 . The network 108 can
the caller to input an urgency class or status before the call 35 include one or more of voice networks and data networks.
or message is routed to the user. In yet another embodiment, For example , one network is a data network providing a
different urgency classes could be defined by the caller. slow speed data channel for transmission of Short Message
Different embodiments or implementations may yield Service (SMS) messages (which are typically limited to 160
different advantages . One advantage is that communications text characters) to a Short Message Service Center (SMSC )
for users of electronic devices can be flexibly provided . 40 and then forwarded on to the destination . Besides short
Another advantage is that communication mode changes can messages (e . g ., SMS messages ), the network 108 can also
be performed at an electronic device to better suit the needs support other messaging protocols for sending and receiving
or condition of the electronic device or user preferences. In enhanced messages (EMS), multimedia messages (MMS),
still another advantage , a user can provide feedback to a email and fax messages . Other networks support faster data
caller without answering a voice call from the caller. 45 channels and voice channels , such as GPRS , UMTS, G4 ,
Another aspect of the invention pertains to improved GSM , CDMA and various protocols , such as UDP, TCP,
approaches to respond to an incoming text message . The WAP, PDP other protocols.
improved approaches enable a recipient to provide a reply According to one embodiment of the invention , one of the
message to an initiator. The incoming text message can be communication devices 102- 106 can send a shortmessage to
presented to the recipient with an audio or textual presen - 50 another of the communication devices 102 - 106 . The short
tation . Thereafter, a reply textmessage can be sent back to message can be text -based or audio - based . The sending
the initiator. The recipient can form the reply textmessage communication device allows its user to create the short
by recording a brief audio message or entering a text message as the user desires and / or as the device permits . For
message . In the case in which a brief audio message is used , example , the user might interact with a keypad or keyboard
the audio message can be automatically converted to a text 55 to enter the short message , or the user might record audio
message before being transmitted to the initiator. inputs (e . g ., speech ) for the short message . The short mes
Further, the computer- implemented methods and systems sage can then be sent to the receiving communication
discussed above can be used in conjunction with one or more device . The sending of the short message may involve
of the various approaches discussed in U . S . patent applica - converting the short message from an audio message to a
tion Ser. No . 11 /006 , 343 . For example, the automated 60 text message, or vice versa . Also , the receiving communi
actions or decisions ( e . g ., intelligent secretary , decision 204 c ation device can further convert the short message from
in FIG . 2 , etc .) of U . S . patent application Ser. No . 11 /006 , audio - to -text or from text-to - audio . In any case , the short
343 can be automatically made by the systems/methods message is presented (e .g ., displayed or played ) to the user
described above. Still further, the various approaches dis - of the receiving communication device. The presentation
cussed in U .S . patent application Ser. No. 11/006 ,343 can be 65 can vary as the user desires or as the device permits.
used in conjunction with one or more the various methods One aspect of the invention pertains to improved
systems discussed above . For example , the systems/methods approaches to respond to incoming voice calls. The
US 10 , 142 ,810 B2
10
improved approaches enable a called party (i.e., a party to decide whether to answer the incoming voice call. The
being called ) to provide some information to a calling party information can , for example, include one or more of name
without directly engaging in a voice call with the calling individual or business), telephone number, or other caller
party . The called party can choose not to take the voice call identification . The information could also include status
from the calling party . Instead , the called party can provide 5 information of the calling party, such as position , health ,
the calling party with some limited information . The limited mood , etc . As an example , the information could be pre
information can be provided in an audio or textual format. sented to the user prior to the decision 204 of the personal
In one embodiment, the limited information provides the call response process 200 shown in FIG . 7 .
calling party with feedback as to why the voice call was not In still another embodiment, an automated decision pro
taken .
FIG . 7 is a flow diagram of a personal call response cess ( e . g .
to decide whether to answer a call can be based on time
, decision 204 ) . For example , the called party can
process 200 according to one embodiment of the invention . previously set a rule , such as that from midnight to 6 am , the
The personal call response process 200 is performed by an
electronic device , such as a mobile communication device party does not want to answer voice calls . Then , during this
( e . g ., mobile telephone ). The personal call response process 15 time period , the electronic device can automatically decide
200 begins with a decision 202 that determines whether not to answer incoming calls. In one implementation , when
there is an incoming voice call . When the decision 202 the electronic device decides not to answer incoming calls,
determines that there is no incoming voice call , then the no indication of incoming calls willbe provided to the called
personal call response process 200 awaits such a call. Once party . For example, from midnight to 6 am , the device would
the decision 202 determines that there is an incoming voice 20 not produce any ring tone. Additionally , if desired , the called
call, a decision 204 determines whether the incoming voice party can also configure the electronic device to automati
call is to be answered . Typically , the user of the electronic cally provide an audio message or a text message to the
device would signal the electronic device as to whether or calling party (e.g ., I'm asleep call me tomorrow ” ).
not to answer the incoming voice call . Alternatively , the FIG . 8 is a flow diagram of an audio message response
electronic device could automatically decide whether to 25 process 300 according to one embodiment of the invention .
answer the call . The audio message response process 300 is , for example ,
When the decision 204 determines that the user desires suitable for use as the processing carried out by block 214
the incoming voice call to be answered , the incoming voice illustrated in FIG . 7 .
call is answered 206 and the user engages 208 in a voice call The audio message response process 300 initially answers
with the calling party . A decision 210 then determines 30 30 302 the incoming voice call. In this operation, the incoming
whether the call has ended . When the decision 210 deter voice call is answered 302 but not in a traditional way .
mines that the call has not yet ended, then the personal call Instead , the electronic circuitry associated with a mobile
response process 200 can return to repeat the block 208
while the voice call continues . Once the decision 210 communication device ( e.g ., mobile telephone ) that receives
determines that the voice call has ended , then the personal 35 the
35 the incoming voice call operates to answer the incoming
voice call for purposes of an audio message response . For
call response process 200 ends .
When the decision 204 determines that the user does not example , a voice channel is established between the calling
desire to answer the incoming voice call , a decision 212 party and the mobile communication device , but the speaker
determines whether the user desires to provide an audio and microphone of the mobile communication device are
message to the calling party. When the decision 212 deter - 40 disabled . In effect , in such an embodiment, neither the called
mines that the user does desire to provide an audio message party nor the calling party perceives that the voice calling
to the calling party , an audio message is obtained and sent has been answered .
214 to the calling party (caller ). Next, one or more predetermined audio messages can be
Alternatively , when the decision 212 determines that the presented 304 by the mobile communication device . The
user does not desire to provide an audio message , a decision 45 presentation 304 of the one or more predetermined audio
216 determines whether the user desires to provide a text messages can , for example , be achieved by audio or visual
message to the calling party . When the decision 216 deter - means. For example , the predetermined audio messages can
mines that the user desires to provide a text message to the be audio output to a speaker associated with the mobile
calling party , a textmessage is obtained and sent 218 to the communication device for the called party or can be visual
calling party . 50 output ( e . g ., text) to a display of the mobile communication
Still further, when the decision 216 determines that the device for the called party (e . g ., user of the mobile commu
user does not desire to provide a text message to the calling nication device ).
party , a decision 220 determines whether the incoming voice A decision 306 then determines whether a predetermined
call is to be directed to voice mail. When the decision 220 audio message has been selected . Here , the decision 306
determines that the incoming voice call should be directed to 55 determines whether the user (i.e ., called party ) of the mobile
voice mail, then the incoming voice call is directed 222 to communication device has selected one or more of the
voice mail. On the other hand , when the decision 220 predetermined audio messages . When the decision 306
determines that the incoming voice call is not to be directed determines that a predetermined audio message has been
to voice mail, the incoming voice call is dropped 224 . selected , then the selected audio message is played 308 for
Following the blocks 214 , 218 , 222 and 224 , the personal 60 the calling party . Here , the mobile communication device
call response process 200 is complete and ends . can output the selected audio message to the calling party
In another embodiment, a personal call response process over the voice channel. Typically, the mobile communica
could announce the calling party to the called party (user). tion device of the called party would not produce an audible
In announcing the calling party , the personal call response output at the mobile communication device , so that the
process would present the called party with information 65 called party would not be disturbed by the sending of the
pertaining to the calling party (e . g ., display or audio sound) .
audio response . The predetermined audio messages are
Such information could , for example , help the called party normally shortmessages (e. g., notmore than 160 characters )
US 10 , 142 ,810 B2
12
so that the duration of time the voice channel is needed FIG . 9 is a flow diagram of a text message response
and/ or the amount of network bandwidth consumed is process 400 according to one embodiment of the invention .
minimal. The text message response process 400 is, for example ,
On the other hand , when the decision 306 determines that processing performed by the block 218 illustrated in FIG . 7 .
none of the predetermined audio messages have been 5 The textmessage response process 400 initially drops 402
selected , then a decision 310 determines whether a custom the incoming voice call. Here, the information to be supplied
audio message is requested . A custom audio message is an to the calling party is a short textmessage ; therefore, there
audio message that is specifically provided for the calling is Next no need for a voice channel.
party . When the decision 310 determines that a custom audio 10 played , 404 one or more predetermined text messages are dis
. Here , the one or more predetermined text
message is not being requested , then the audio message messages would
response process 300 returns to repeat the decision 306 and associated with normally the mobile
be displayed on a display screen
communication device being
subsequent operations. Alternatively , when the decision 310 utilized by the called party . A decision 406 then determines
determines that a custom audio message is requested , then a whether one (or more ) of the predetermined text messages
custom audio message is recorded 312 312 .; Thereanter
Thereafter ,, the
the 15 has been selected . When the decision 406 determines that a
custom audio message that has been recorded can be played predetermined text message has been selected , then the
314 for the calling party (caller ). Here , typically , the custom selected text message is transmitted 408 to the caller (i.e .,
audio message would be output by the mobile communica the calling party ).
tion device of the called party over the voice channel to the On the other hand , when the decision 406 determines that
calling party. Typically , the mobile communication device of 20 a predetermined text message has not been selected , then a
mobile communication device , so that the called party requested . When the decision 410 determines that a custom
would not be disturbed by the sending of the audio response . text message is not requested , then the text message
sages (e . g ., not more than 160 characters ) so that the 25 subsequent operations. Alternatively , when the decision 410
duration of time the voice channel is needed and / or the determines that a custom textmessage is requested , then the
amount of network bandwidth consumed is minimal. custom text message is entered 412 . Here, the called party
Following the operations 308 and 314 , the incoming voice interacts with the mobile communication device to enter the
call is closed 316 . In other words, after the selected audio custom text message . Then , the custom text message is
message or the custom audio message is played 308, 314 , the 30 transmitted 414 to the caller. In one embodiment , the trans
incoming voice call can be closed 316 . Following the block mission 408 , 414 of the textmessage can be performed over
316 , the audio message response process 300 is complete a communication network , such as a network having a Short
and ends. Message Service Center ( SMSC ) supporting Short Message
to a called party can be determined in a static or dynamic 35 the selected text message or the transmission 414 of the
manner. A static determination would , for example , be when custom textmessage , the text message response process 400
the called party has previously set or recorded an audio is complete and ends .
message to be utilized . Typically , with static determination , An alternative embodiment of a text message response
the list of audio messages remains the same (i.e ., static ) until process could operate to answer the incoming voice call and
changed ( e . g ., by the called party ). A dynamic determination 40 announce to the caller that a text message will be forthcom
would allow the audio messages in the list ( or the ordering ing. Then , the incoming voice call could be promptly
of the audio messages in the list) to change without specific dropped . This additional operation could , for example , be
action by the user or the called party . For example , the list used with the text message response process 400 by pro
settings, configuration information , or prior usage . Prior 45 trated in FIG . 9 .
usage can include biasing the list of audio messages such The predetermined text messages being displayed 404 to
that thosemessages being most often selected appear higher a called party can be determined in a static or dynamic
in the list. The list or ordering of the audio messages can also manner . A static determination would , for example , be a text
depend on the calling party, type of calling party , location of message the called party has previously set or entered .
calling party or called party , and the like. The list of audio 50 Typically, with static determination , the list of text messages
messages can be represented by text and /or graphics ( e.g., remains the same (i.e., static ) until changed (e.g ., by the
icons) . called party ) . A dynamic determination would allow the text
The audio message response process 300 flexibly enables messages in the list (or the ordering of the text messages in
a user to either select one or more predetermined audio the list) to change automatically , and not by the user. For
messages or provide a custom audio message to be used as 55 example , the list or ordering of the textmessages can depend
an audio message response to a calling party . However, it on preference settings, configuration information , or prior
should be recognized that, in other embodiments , an audio usage. To illustrate , prior usage can include biasing the list
message response process can alternatively simply pertain to of text messages such that those messages being most often
selection of a predetermined audio message . Further , in still 60 text messages can also depend on the calling party , type of
other embodiments , an audio message response process can calling party , location of calling party or called party , and the
first determine whether a custom audio message is to be like . The list of text messages can identify each textmessage
provided before presenting predetermined audio messages . with text ( e.g., at least a portion of the corresponding text
In yet other embodiments, an audio message response pro - message , or an abbreviation ) and/ or graphics ( e . g ., icons).
cess can answer the incoming voice call later in the pro - 65 The text message response process 400 flexibly enables a
cessing than operation 302 as shown in FIG . 8 (e . g ., before user to either select one or more predetermined text mes
operations 308 and 314 ). sages or provide a custom text message to be used as a text
US 10 ,142 ,810 B2
13 14
message response to a calling party . However, it should be text messages presented to Tom can be dependent on some
recognized that, in other embodiments, a text message criteria (i.e ., automatically selected ). Alternatively , it is
response process can alternatively simply pertain to only possible that Tom might want to edit the predetermined text
providing a custom text message, or only permitting selec message, such can be permitted . Asyet another example, the
tion of a predetermined text message . Further, in still other 5 text message can embed dynamic information , such as
embodiments, a text message response process can first position , e . g ., “ I' m in [ position ] now , so I' ll get back to you
determine whether a custom textmessage is to be provided later.” The position can be determined using a GPS receiver
before presenting predetermined text messages in the mobile phone or acquired by a remote computer and
FIG . 10 is a flow diagram of an automated call response provided to the mobile phone. The position may also be
process 500 according to one embodiment of the invention . 10 further processed ( locally or remotely ) into a more user
The automatic call response process 500 is substantially friendly form , such as city, school, restaurant name, or street
similar in many ways to the personal call response process type addresses . The position could also be used above to
200 illustrated in FIG . 7 . However, the automated call assist the user in deciding whether to answer the incoming
response process 500 operates to reduce user input at the call or decline to answer the call.
mobile communication device by making use of stored data 15 7 . If hardware components , configuration or preferences
pertaining to its hardware components , configuration or are taken into consideration , as illustrated in FIG . 10 , the
preferences. In this regard , the automatic call response above scenario can be modified . For example , if Tom is
process 500 includes a decision 502 that determines whether using a head - set with his mobile phone, then an audio
a head - set is active. When the decision 502 determines that message may be most convenient, assuming that Tom wants
a head - set is active , then the automatic call response process 20 to provide a particular (i. e ., customized ) message to Bill.
500 can prefer, suggest or require the user to obtain and send The head -set allows Tom to record a brief audio message .
214 an audio message to the caller in response to an Less conveniently , the head - set can be used to present a list
incoming voice call . Alternatively , when the decision 502 of predetermined audio messages and allow Tom ' s selection
determines that a head -set is not active , then a decision 504 therefrom by a button or voice -command .
can determine whether a display is present. In other words, 25 8 . If Tom is not using a head - set, then a text message
the decision 504 can determine whether the mobile com - response might be more suitable. This would typically
munication device has a display . When the decision 504 require that Tom ' s mobile phone have a display and a
determines that the mobile communication device does have keypad . Even so , without a head -set, Tom could still record
a display , then themobile communication device can operate an audio message , though such would likely be less conve
to obtain and send 218 a text message to the caller. Of 30 nient.
course , this assumes that the caller can support text mes 9 . Tom can also not provide an audio message or a text
sages even though they initially called with a voice call . message and simply let the incoming call roll -over into
Hence, in another embodiment, the automatic call response voice mail.
process can operate to query or obtain information regarding The exemplary scenario can also be used in a case where
the caller ' s communication device capabilities. 35 the called party is using one line but the mobile device has
An exemplary scenario of how the previously described multi-line capabilities or call waiting. In such case , the
automatic call response process could work according to one mobile phone can enable the called party to provide a brief
implementation is as follows : audio or text message to the calling party as noted above.
1 . From his mobile phone , Bill calls Tom 's mobile phone. Alternatively, the mobile phone can itself automatically (i.e.,
2 . Tom is alerted by his mobile phone of an incoming call. 40 without user input ) respond to the calling party via an audio
Optionally , caller information (i. e ., pertaining to Bill) can be or text message since the mobile phone is aware that the
displayed or announced to Tom . called party is on the other line .
3 . Tom can choose to answer the incoming call or decline In this aspect of the invention , the calling party and the
to answer the call. called party often use mobile communication devices , such
4 . In the event that Tom declines to answer the call, Tom 45 as mobile phones . However, the parties can alternatively use
can have the opportunity to provide the caller with a brief other electronic devices , such as a PDA , a computer, etc .
audio or text message . Further, the option to provide a text response could be
5 . If an audio message is to be provided , then Tom can prevented if the caller 's device is known to not support text
either record a personalized message or select one of a messages .
plurality of predetermined audio messages . In this case , the 50 The advantages of the previously described embodiments
incoming call is answered by Tom 's mobile phone and then are numerous. Different embodiments or implementations
the audio message is played for the caller, thereafter the call may yield different advantages . One advantage is that com
is dropped . The audio messages are typically brief ( i.e ., munications for users of electronic devices can be flexibly
short ), and examples of audio messages are : (i) “ Will call in provided . Another advantage is that communication mode
10 minutes ,” (ii) “ Cannot talk now ," (iii ) “ I' m in a meeting ,” 55 changes can be performed at an electronic device to better
or (iv ) “ Please don 't call anymore.” suit the needs or condition of the electronic device or user
6 . On the other hand, if a text message is to be provided , preferences . In still another advantage , a user can provide
then Tom can either enter a personalized text message or feedback to a caller without answering a voice call from the
select from a plurality of predetermined text messages . In caller.
this case, the incoming call is dropped , and the entered text 60 Another aspect of the invention pertains to improved
message or the selected one of the predetermined text approaches to respond to an incoming text message . The
messages is sent . Examples of text messages are : (i) “ Will improved approaches enable a recipient to provide a reply
call in 10 minutes,” (ii) “ Cannot talk now ,” ( iii) “ I'm in a message to an initiator. The incoming text message can be
meeting ," or ( iv ) “ Please don ' t call anymore .” The text presented to the recipient with an audio or textual presen
messages can be English ( or other language ) words or 65 tation . Thereafter , a reply text message can be sent back to
phrases, or can be condensed text strings (e . g ., such as slang the initiator. The recipient can form the reply textmessage
or chat language ). In one embodiment, the predetermined by recording a brief audio message or entering a text
US 10 ,142 ,810 B2
15 16
message . In the case in which a brief audio message is used , small form factor and price competition , tend to have limited
the audio message can be automatically converted to a text processing capability. Accordingly , in one embodiment,
message before being transmitted to the initiator. text-to -speech conversion can be off - loaded from themobile
FIG . 11 is a flow diagram of a message presentation device . For example , a remote server computer can be
process 600 according to one embodiment of the invention . 5 provided the textmessage and produce the resulting audio
The message presentation process 600 is performed by an message , and then supply the audio message to the mobile
electronic device , such as a mobile communication device . device . The remote server computer can be a networked
The message presentation process 600 begins with a server coupled to the network 108 . One example of a
decision 602 that determines whether an incoming text networked server is a gateway computer for a wireless
message is present . Typically , the incoming text message 10 electronic device , such as a mobile telephone.
would be transmitted to the mobile communication device FIG . 12 is a flow diagram of a reply message process 700
from another communication device. When the decision 602 according to one embodiment of the invention . The reply
determines that an incoming text message is not present, message process 700 is performed by an electronic device ,
then the message presentation process 600 awaits such such as a mobile communication device .
message . Once the decision 602 determines that an incom - 15 The reply message process 700 begins with a decision 702
ing textmessage has been received , a decision 604 deter - that determines whether a reply message is to be sent.
mines whether an audio or text presentation is to be utilized . Typically, the reply message process 700 follows the pre
The decision 604 can be performed in a variety of different sentation of an incoming text message to a user of a mobile
ways . For example, the determination of whether to utilize communication device. Hence , the reply message to be sent
an audio or text presentation can be based on user input or 20 is a reply to the incoming text message. However, in other
can be automatically determined through a use of configu - embodiments , the reply message to be sent can be merely an
ration or preference information or hardware components initial message as opposed to a response to an earlier
(e .g ., display, speaker, head -set ). message .
When the decision 604 determines that an audio presen - In any case , when the decision 702 determines that a reply
tation is to be utilized , the incoming text message is con - 25 message is not to be sent, then the reply message process 700
verted 606 to an audio message. For example , a text -to - ends or simply awaits the need to send a reply message . On
speech conversion can be performed . In one embodiment, a the other hand, when the decision 702 determines that a
user of the electronic device can be permitted to choose reply message is to be sent, then a decision 704 determines
speech characteristics, such as a voice, tone , pace, accent, or whether an audio or text message is to be formed . The
mood , for the resulting speech . For example , a user could 30 decision 704 can be performed in a variety of different ways .
choose speech characteristics by preference settings . In For example , the determination of whether to send an audio
another embodiment, the incoming textmessage can include or text message can be based on user input or can be
or reference speech characteristics so that the initiator can automatically determined through a use of configuration or
control or influence speech characteristics. In still another preference information or hardware components (e .g ., dis
embodiment, if the text to be converted contains condensed 35 play, speaker, head - set ).
text (e .g., such as slang or chat language ), the resulting When the decision 704 determines that an audio message
speech can pertain to an uncondensed form of the text. The is to be formed , then the reply message process 700 prompts
ability to convert from condensed text to resulting speech for 706 for an audio message . Here, the prompt 706 can be
uncondensed text can be facilitated by pattern matching. For directed to the user of the mobile communication device .
example , in chat language “ LOL ” can be converted to an 40 The prompt can be an audio or textual indication . Next, a
audio message for “ lots of love ." In one implementation , a decision 708 determines whether an audio message has been
table can store audio messages corresponding to chat terms recorded . When the decision 708 determines that the audio
or phrases . In another implementation , a first table would message has not been recorded , then the reply message
store uncompressed terms or phrases corresponding to chat process 700 awaits the audio message . Once the decision
terms or phrases, and a second table would store audio 45 708 determines that the audio message has been recorded ,
messages corresponding to the uncompressed terms or then the audio message is converted 710 to a text message .
phrases . In one embodiment , if the audio message recorded is greater
After the incoming text message is converted to the audio than a maximum text message size ( e . g ., 150 or 160 char
message , the audio message is played 608. Typically , the acters ), then the audio message can be shortened so that the
audio message is played 608 by the mobile communication 50 resulting text message does not exceed the maximum text
device for the user . For example , the audio message can be message size . One way to shorten the textmessage is to use
output to a speaker of the mobile communication device or abbreviations. For example , the words “ For example ” can be
a headset used therewith . As a result , the user of the mobile changed to " e . g .” . Such conversion can be again be per
wireless communication device receives an audio message formed by matching entries in tables . Another way to
even though the incoming message was a textmessage . 55 shorten is to remove non - essential text. Still another way to
On the other hand, when the decision 604 determines that shorten is to clip off or truncate the text message at the
a text presentation is to be utilized , the incoming text maximum text message size . In another embodiment, the
message is displayed 610 . Here , the incoming text message resulting text message might provide an indication that it
would be displayed 610 on a display associated with the was converted from an audio message . Following the block
mobile communication device . Following the blocks 608 60 710 , the text message is transmitted 712 over a wireless
and 610, the message presentation process 600 ends. network .
As discussed above , text-to -speech conversion can be Alternatively, when the decision 704 determines that a
invoked and performed on an electronic device, which may text message is to be formed , then a text entry screen is
be a mobile communication device . While text- to - speech displayed 714 . Next, a decision 716 determines whether a
conversion , particularly if high quality is desired , requires 65 text message has been entered . When the decision 716
substantial processing capabilities, mobile electronic determines that a text message has not yet been entered , then
devices, such as mobile communication devices, given their the reply message process 700 awaits entry of the text
US 10 ,142 ,810 B2
18
message . Once the text message has been entered , the text flexibly provided . Another advantage is that communication
message is transmitted 712 over the wireless network . mode changes can be performed at an electronic device to
Following the block 712 , the reply message process 700 better suit the needs or condition of the electronic device or
ends. user preferences . Still another advantage is that conversion
Although the reply message process 700 provides for the 5 of an audio message to a text message facilitates use a low
user to enter a custom text or audio message, it should be cost network ( such as the SMS network ). Another advantage
understood that the reply message can alternatively be is reduced network bandwidth load . Yet still another advan
formed through use of semi- custom or predetermined reply tage is that the sender can get back a message in the same
messages from which the user of the mobile communication format as they sent the originalmessage , though the recipi
device can choose. The use of semi- custom or predeter - 10 ent may use the message in a different format or mode ( e. g .,
mined reply messages can be achieved as noted above in a recipient hears the text message as an audio message ).
number of embodiments , and can serve to simplify the Moreover, it should be noted that with regards to any of
An exemplary scenario of how message presentation and incoming to an electronic device, not only can the user of the
reply message processes could work according to one imple - 15 mobile device take an action ( e . g ., button press or voice
mentation of the second aspect is as follows: command ) to decline the call/message but also the electronic
1. From his mobile phone, Bill prepares and sends a text device itself can automatically decline the call/message such
message to Tom 's mobile phone . that the user is not disturbed . For example, an electronic
2 . Tom is alerted by his mobile phone of an incoming text device can be configured through user settings (e.g ., pref
message , such as by displaying at least a portion of the text 20 erences) to decline calls/messages matching certain criteria .
message and /or otherwise notifying Tom of the text mes Also , an auto reply message can be configured to be auto
sage . matically sent in response to the call/message. For a known,
3. Tom 's mobile phone can decide whether to present the undesired marketing caller /message sender , the electronic
text message on a display screen of Tom 'smobile phone, or device can automatically send a reply message demanding
to first convert the text message to an audio message and 25 the sender not to call or send messages anymore , and to
then present the audio message to Tom ( e. g ., play the audio remove your information from their database .
message ). Of course , Tom can interact with Tom ' s mobile Text messages received or sent can optionally embed
present the message . textmessage be converted to an audio format. The speech
4 . Thereafter, if desired , Tom can prepare and send a reply 30 characteristics can pertain to voice , tone, pace, accent,
message back to Bill. This reply message can be prepared and /or mood . The speech characteristics for the resulting
initially as a text message or an audio message . Tom 's speech can be set in preference or configuration information ,
mobile phone and/ or Tom can determine whether the reply set on a permessage basis by users , or set by evaluation of
message is initially prepared as a textmessage or as an audio monitored data pertaining to the user.
message . If an audio message is initially created , such audio 35 Additionally, the messages being transmitted can be
message must be converted to a text message prior to encrypted for security purposes .
transmission . Eventually, the reply message is sent to Bill as in one embodiment, an electronic device performing
a text message . Tom ' s mobile phone can assist with the communications using audio and /or text messages according
custom or predetermined reply message from which Tom 40 thereto ) one or more electrical components for enhancing
and /or Tom 's mobile phone can choose . the hearing of the user of the electronic device . The elec
5 . If Tom is using a head - set with his mobile phone, then tronic device will normally include a microphone and a
Tom wants to provide a particular (i.e ., customized ) message an automated secretary for a user of an electronic device .
to Bill. The head -set allows Tom to easily record a brief 45 The automated secretary can completely or partially respond
audio message . Less conveniently, the head -set can be used to an incoming call /message so as to reduce disturbances to
to present a list of predetermined audio messages and allow the user. The user can personalize the automated secretary
Tom ' s selection therefrom by a button or voice - command . through user settings ( e . g ., preferences ), or the automated
6 . If Tom is not using a head -set, then a text message secretary can learn over time how to handle different incom
response might be more suitable . This would typically 50 ing calls/messages . Besides handling or assisting the user
require that Tom 's mobile phone have a display and a with incoming calls /messages, the automated secretary can
keypad . Even so , without a head - set, Tom could still record also assist with other activities , such as making calendar
an audio message, though such would likely be less conve - entries ( e . g ., meetings ) in a calendar or responding to
7 . Tom can also not provide a reply message and simply 55 taining to the user's schedule as maintained by the calendar
not respond to the incoming text message. Alternatively, (though the user could restrict such access to certain infor
Tom can configure his mobile phone to automatically pro - mation and /or inquiring parties ). For example , if an incom
duce and send a reply message based on user settings or ing text message asks " available for lunch today ?”', the
preferences, position , configuration , status, etc . automated secretary can check the user ' s availability for
In this aspect of the invention , the calling party and the 60 lunch by way of the user ' s calendar, then if the user is not
called party often use mobile communication devices , such available the automated secretary can quickly informing the
as mobile phones. However, the parties can alternatively use inquiring party of same or propose another date . On the
other electronic devices , such as a PDA , a computer, etc. other hand , if the lunch time period is available in the user' s
The advantages of the invention exemplified by FIGS. calendar, then the automated secretary can either directly
11 - 12 are numerous. Different embodiments or implemen - 65 respond to the inquiring party of acceptance or propose a
tationsmay yield different advantages . One advantage is that response to the user for review , modification and /or trans
communications for users of electronic devices can be mission .
US 10 , 142 ,810 B2
19 20
It should be obvious to those skilled in the art that a What is claimed is :
number of embodiments performing communications using 1 . A computer-implemented method for managing elec
voice as well as audio and/ or text messages can be imple - tronic communications using at least a network -based portal
mented using voice over Internet Protocol technologies, at least based on Internet protocol, the method comprising :
with signals delivered over the Web . For example , a calling 5 providing a plurality of communication options to a first
party 's communication or mobile device can include an user to be selected as a selected option of communi
adapter to convert voice signals to data packets before cation for a message from the first user to a second user
sending them over the Internet. A service provider can via an electronic device associated with the second
convert the packets back into voice signals before sending user, with the first user being identified at least depend
the voice signals to the called party 's communication ing on a prior registration process by the first user
device . Similarly , embodiments can be implemented using regarding the use of the network -based portal, and with
the plurality of communication options provided to the
voice over wireless protocols , such as Wi-Fi or Wi-Max first user to send messages to the electronic device
networks. Using such technologies , computing devices can associated with the second user,
become communication devices. 15 wherein the plurality of communication options include
The various embodiments , implementations, features and textmessaging and voice communication , and
aspects of the invention noted above can be combined in wherein all of the communication options use one
various ways or used separately. Those skilled in the art will identifier associated with the second user for the
understand from the description that the invention can be second user to receive messages , at least in view of
equally applied to or used in other different settings with 20 the network -based portal being based on the Internet
respect to various combinations, embodiments , implemen protocol;
tations or features provided in the description
tion herein
herein . receiving an indication regarding one of the plurality of
The invention can be implemented in software , hardware communication options, via the network - based portal,
or a combination of hardware and software . A number of from an electronic device associated with the first user,
embodiments of the invention can also be embodied as 25 the indication indicating the selected option of com
computer readable code on a computer readable medium . munication for the message from the plurality of com
The computer readable medium is any data storage device munication options provided ;
that can store data which can thereafter be read by a permitting the second user to block the first user from
computer system . Examples of the computer readable reaching the second user via the network - based portal;
medium include read -only memory , random - access 30 and
memory , CD -ROMs, magnetic tape, optical data storage enabling, via the network -based portal, the message to be
devices, and carrier waves . The computer readable medium received by the second user through the electronic
can also be distributed over network -coupled computer device associated with the second user, using the
systems so that the computer readable code is stored and selected option of communication , based on the one
executed in a distributed fashion . 35 identifier associated with the second user, in view of the
Numerous specific details are set forth in order to provide second user not blocking the first user from reaching
a thorough understanding of the present invention . However, the second user , wherein a piece of information regard
it will become obvious to those skilled in the art that the ing the second user blocking the first user from reach
invention may be practiced without these specific details . ing the second user is stored in a storage medium if the
The description and representation herein are the common 40 second user has blocked the first user from reaching the
meanings used by those experienced or skilled in the art to second user, with the piece of information being based
most effectively convey the substance of their work to others on at least an input previously submitted by the second
skilled in the art. In other instances , well -known methods, user,
procedures, components , and circuitry have not been wherein the method comprises determining availability of
described in detail to avoid unnecessarily obscuring aspects 45 the second user,
of the present invention . wherein the method requires contact information associ
In the foregoing description , reference to " one embodi ated with the second user to allow the second user to
ment” or “ an embodiment” means that a particular feature , receive messages via the network -based portal,
structure , or characteristic described in connection with the wherein even when the message is received by the second
embodiment can be included in at least one embodiment of 50 user through the electronic device associated with the
the invention . The appearances of the phrase " in one second user based on the one identifier associated with
embodiment” in various places in the specification are not the second user, the contact information associated with
necessarily all referring to the same embodiment, nor are the second user is not provided via the network -based
separate or alternative embodiments mutually exclusive of portal to the first user through the electronic device
other embodiments. Further, the order of blocks in process 55 associated with the first user, and
flowcharts or diagrams representing one or more embodi- wherein the one identifier associated with the second user
ments of the invention do not inherently indicate any par is distinct from the contact information associated with
ticular order nor imply any limitations in the invention . the second user .
The many features and advantages of the present inven - 2 . A computer -implemented method as recited in claim 1 ,
tion are apparent from the written description and , thus, it is 60 wherein the plurality of communication options include
intended by the appended claims to cover all such features multimedia messaging using the one identifier associated
and advantages of the invention . Further, since numerous with the second user for the second user to receivemessages ,
modifications and changes will readily occur to those skilled at least in view of the network -based portal being based on
in the art, it is not desired to limit the invention to the exact the Internet protocol.
construction and operation as illustrated and described . 65 3. A computer- implemented method as recited in claim 1 ,
Hence , all suitable modifications and equivalents may be wherein the plurality of communication options include
resorted to as falling within the scope of the invention . group messaging using the one identifier associated with the
US 10 , 142 ,810 B2
21 22
second user for the second user to receive messages , at least identifier associated with the second user, in view of the
in view of the network -based portal being based on the second user not blocking the first user from reaching
Internet protocol. the second user, wherein a piece of information regard
4 . A computer- implemented method as recited in claim 3 , ing the second user blocking the first user from reach
wherein the one identifier associated with the second user 5 ing the second user is stored in a storage medium if the
includes a digital identity of the second user. second user has blocked the first user from reaching the
5 . A computer -implemented method as recited in claim 4 , second user, with the piece of information being based
wherein the contact information associated with the second on at least an input previously submitted by the second
user includes at least one of a phone number or an email user,
address of the second user. 10 wherein the instructions, when executed , cause the com
6 . A computer -implemented method as recited in claim 5 , puting apparatus to determine availability of the second
wherein the electronic device associated with the second user, and to require contact information associated with
user is a wireless electronic device , and the second user to allow the second user to receive
wherein the electronic device associated with the first user messages via the network -based portal,
is a wireless electronic device . 15 wherein even when the message is received by the second
7 . A computer- implemented method as recited in claim 4 , user through the electronic device associated with the
wherein the enabling the message to be received by the second user based on the one identifier associated with
second user depends on a time. the second user, the contact information associated with
8. A computer -implemented method as recited in claim 7 , the second user is not provided via the network - based
wherein the enabling the message to be received by the 20 portal to the first user through the electronic device
second user depends on a period of time. associated with the first user, and
9 . A computer- implemented method as recited in claim 8 , wherein the one identifier associated with the second user
wherein themethod comprises not presenting the message to is distinct from the contact information associated with
the second user depending on the period of time. the second user.
10 . A computer- implemented method as recited in claim 25 12 . A computing apparatus as recited in claim 11 , wherein
4 , wherein the enabling the message to be received by the the plurality of communication options include multimedia
second user enables leaving a voice mail. messaging using the one identifier associated with the sec
11. A computing apparatus for managing electronic com ond user for the second user to receive messages, at least in
munications using at least a network - based portal at least view of the network -based portal being based on the Internet
based on Internet protocol, the computing apparatus com - 30 protocol.
prising: 13 . A computing apparatus as recited in claim 11 , wherein
at least one computing device ; and the plurality of communication options include group mes
one or more storage devices coupled to the at least one saging using the one identifier associated with the second
computing device , with the one or more storage devices user for the second user to receive messages, at least in view
storing instructions that, when executed , cause the 35 of the network -based portal being based on the Internet
computing apparatus to : protocol.
providing a plurality of communication options to a first 14 . A computing apparatus as recited in claim 13 , wherein
user to be selected as a selected option of communi - the one identifier associated with the second user includes a
cation for a message from the first user to a second user digital identity of the second user.
via an electronic device associated with the second 40 15 . A computing apparatus as recited in claim 14 , wherein
user, with the first user being identified by the com - the contact information associated with the second user
puting apparatus at least depending on a prior registra - includes at least one of a phone number or an email address
tion process by the first user regarding the use of the of the second user .
network -based portal, and with the plurality of com - 16 . A computing apparatus as recited in claim 15 , wherein
munication options provided to the first user to send 45 the enabling the message to be received by the second user
messages to the electronic device associated with the includes not presenting the message to the second user
second user, depending on a period of time.
wherein the plurality of communication options include 17 . A computing apparatus as recited in claim 15 ,
text messaging and voice communication , and wherein the electronic device associated with the second
wherein all of the communication options use one 50 user is a wireless electronic device , and
identifier associated with the second user for the wherein the electronic device associated with the first user
second user to receive messages , at least in view of is a wireless electronic device .
the network -based portal being based on the Internet 18 . A computing apparatus as recited in claim 14 , wherein
protocol; the enabling the message to be received enables leaving a
receiving an indication regarding one of the plurality of 55 voice mail.
communication options, via the network -based portal, 19 . A non -transitory computer readable medium including
from an electronic device associated with the first user, at least executable computer program code stored therein for
the indication indicating the selected option of com - managing electronic communications using at least a net
munication for the message from the plurality of com - work -based portal at least based on Internet protocol, said
munication options provided ; 60 computer readable medium comprising:
permitting the second user to block the first user from computer program code for providing a plurality of com
reaching the second user via the network -based portal; munication options to a first user to be selected as a
and selected option of communication for a message from
enabling, via the network - based portal, the message to be the first user to a second user via an electronic device
received by the second user through the electronic 65 associated with the second user, with the first user being
device associated with the second user, using the identified at least depending on a prior registration
selected option of communication , based on the one process by the first user regarding the use of the
US 10 ,142,810 B2
23 24
network -based portal, and with the plurality of com computer program code for determining availability ofthe
munication options provided to the first user to send second user; and
messages to the electronic device associated with the computer program code for requiring contact information
second user , associated with the second user to allow the second
wherein the plurality of communication options include 5 user to receive messages via the network -based portal,
text messaging and voice communication , and wherein even when the message is received by the second
wherein all of the communication options use one user through the electronic device associated with the
identifier associated with the second user for the second user based on the one identifier associated with
second user to receive messages, at least in view of the second user, the contact information associated with
the network -based portal being based on the Internet 10 the second user does not have to be provided to the first
protocol;
computer program code for receiving an indication user through the electronic device associated with the
regarding one of the plurality of communication first user,
options, via the network -based portal, from the elec wherein the one identifier associated with the second user
tronic device associated with the first user, the indica - 15 is distinct from the contact information associated with
tion indicating the selected option of communication the second user,
for the message from the plurality of communication wherein the plurality of communication options include
options provided ; multimedia messaging and group messaging , all of
computer program code for permitting the second user to which using the one identifier associated with the
block the first user from reaching the second user via 30 second user for the second user to receive messages, at
the network -based portal; least in view of the network -based portal being based
computer program code for enabling, via the network on the Internet protocol,
based portal, the message to be received by the second wherein the one identifier associated with the second user
user through the electronic device associated with the includes a digital identity of the second user, and
second user, using the selected option of communica - 25 wherein the contact information associated with the sec
tion , based on the one identifier associated with the ond user includes at least one of a phone number or an
second user, in view of the second user not blocking the email address of the second user.
first user from reaching the second user, wherein a 20 . A non -transitory computer readable medium as recited
piece of information regarding the second user block in claim 19 ,
ing the first user from reaching the second user is stored 20 wherein the electronic device associated with the second
in a storage medium if the second user has blocked the user is a wireless electronic device , and
first user from reaching the second user, with the piece wherein the electronic device associated with the first user
of information being based on at least an input previ is a wireless electronic device .
ously submitted by the second user;
EXHIBIT D
US008744407B2
(12) United States Patent (10) Patent No.: US 8,744.407 B2

Cheung et al. (45) Date of Patent: Jun. 3, 2014
(54) SYSTEMS AND PROCESSES TO MANAGE (56) References Cited
MULTIPLE MODES OF COMMUNICATION
U.S. PATENT DOCUMENTS
(75) Inventors: yTong,yeMountain
Ches,Tai pSN rter
View, CA (US); C.
5.425,516 A
5,548,636 A
6/1995 Daines
8/1996 Bannister et al.
Douglass Thomas, Campbell, CA (US) 5,758,079 A 5/1998 Ludwig et al.
5,828,731 A 10/1998 Szlam et al.
(73) Assignee: IpVenture, Inc., Los Altos, CA (US) 5,930,700 A * 7/1999 Pepper et al. .............. 455,435.3
5,970,388 A 10, 1999 W11
: - -- 6,119,022 A 9/2000 Osborn et al.
(*) Notice: Subject to any disclaimer, the term of this 6,327,628 B1 12/2001 Anuffetal.
patent is extended or adjusted under 35 6.463,462 B1 10/2002 Smith et al.
U.S.C. 154(b) by 47 days. 6,577,859 B1 6/2003 Zahavi et al.
6,636,888 B1 10/2003 Bookspan et al.
(21) Appl. No.: 12/798,995 6,665,534 B1 12/2003 Conklin et al.
(Continued)
(22) Filed: Apr. 14, 2010
FOREIGN PATENT DOCUMENTS
(65) Prior Publication Data
CN 1453.981 11/2003
US 2010/0205272 A1 Aug. 12, 2010 WO WO 01/45343 A2 6/2001
Related U.S. Application Data OTHER PUBLICATIONS
(63) Continuation of application No. 1 1/452,115, filed on First Office Action for CN Patent Application No. 2006800279849,
Jun. 12, 2006, now Pat. No. 7,729,688, which is a dated Mar. 26, 2010 (17 pages).
continuation-in-part of application No. 11/006,343, Continued
filed on Dec. 7, 2004, now Pat. No. 7,116,976. (Continued)
(60) Provisional application No. 60/527.565, filed on Dec. Primary Examiner — Kiet Doan
8, 2003, provisional application No. 60/689,686, filed
on Jun. 10, 2005. (57) ABSTRACT
A computer-implemented system and method to manage the
(51) Int. Cl. communication of a user are disclosed. In one embodiment,
H04M I/66 (2006.01) when a person tries to electronically convey a message to the
(52) U.S. Cl. user, the status of the user, the identity of the person, and the
USPC - - - - - - - - - - -grgrrr. 455/411 urgency of the message can be identified. The access priority
(58) Field of Classification Search of the person can be determined based on the person's iden
USPC ........ 455/414, 415, 411,412.1, 412.2, 435.3, tity. Then, the message can be managed using one or more
455/527, 418, 518, 15,422.1, 436; rules and in view of the status of the user, the access priority
709/217, 225; 379/20704, 88.16; of the person and the urgency of the message.
370/444, 336,296, 401, 328, 310,356
See application file for complete search history. 59 Claims, 2 Drawing Sheets
US 8,744.407 B2
Page 2
(56) References Cited Short Message Service/Interactive Voice Response (SMSIVR),

Lucent Technologies, 2003, pp. 1-2.
U.S. PATENT DOCUMENTS “text messaging”, Vodafone Group, 2001, Vodafone—Services, "All
about text messaging”.http://www.vodafone.co.nz/services/07.a. 1
6,788,766 B2 9/2004 Logan two way messaging.jsp?hd=4yourbusiness&..., downloaded Oct.
6,801,793 B1 10/2004 Aarnio et al.
6,816,578 B1 1 1/2004 Kredo et al. 14, 2003, pp. 1-2.
6,819,757 B1 1 1/2004 Cook et al. “We bring relevance to communications.” Cnet News, Ina Fried, Jul.
6,819,945 B1 1 1/2004 Chow et al. 21, 2005, pp. 1-2.
6,978,136 B2 12/2005 Jenniges et al. Appenzeller, et al., “The Mobile People Architecture”. Technical
7,010,288 B2 3, 2006 Brown et al. Report: CSL-TR-00000, Computer Systems Laboratory, Depart
7,010,332 B1 3, 2006 Irvin et al.
7,027,842 B2 4/2006 Zhang et al. ments of Electrical Engineering and Computer Science, Stanford
7,043,261 B2 5, 2006 Krishnan University, Jan. 1999, pp. 1-13.
7,072,452 B1 7/2006 Roberts et al. BlackBerry, “Voice and SMS'. http://www.blackberry.com/prod
7,085,253 B2 8/2006 Yang ucts/service? voices Sms.shtml?DCPID=hmsvoice downloaded Oct.
7,107,010 B2 9, 2006 Heinonen et al. 2, 2003, p. 1.
7,110,963 B2 9/2006 Negreiro
7,111,044 B2 9, 2006 Lee Calsyn, Martin and Desseault, Lisa, “Presence Information Protocol
7,116,976 B2 10/2006 Thomas et al. Requirements.” Internet Draft, Feb. 9, 1998, pp. 1-27.
7,188,073 B1 3/2007 Tam et al. Emergin Inc., “Emergin WirelessOffice 5.0', http://www.emergin.
7,224,775 B1* 5/2007 Shaffer et al. .............. 379.88.16 com/?source=overture, downloaded Oct. 2, 2003, p. 1.
7,376.434 B2 5, 2008 Thomas et al.
7,403,972 B1 7/2008 Lau et al. Fastmobile Inc., “Dialog GSM launches Push 'n' Talk walkie talkie
7,729,688 B2 6/2010 Cheung et al. service Push to Talk over Cellular Now in Sri Lanka Dialog GSM
7,792,552 B2 9, 2010 Thomas et al. Pioneers Latest GSM Advancement', Press Release, Dec. 1, 2004,
7,890,128 B1 2/2011 Thomas et al. pp. 1-2.
8, 112,104 B1 2/2012 Thomas et al. Fastmobile, “fastmobile's fastchat TM Instant Communications Appli
8,280,419 B1 10/2012 Thomas et al. cation is Coming to Thousands of Mobile Phone Retail Stores
2001 0011014 A1 8/2001 Higuchi et al. Nationwide', fastmobile Press Release, Sep. 15, 2003, pp. 1-3.
2001/0O28709 A1 10, 2001 Makela et al.
2001/003.1633 A1 10, 2001 Tuomela et al. IMBOT, Press Release, “IMBOT offers new Text 2 Voice Service
2002fOO67806 A1 6/2002 Rodriguez et al. Text 2 Voice service enables wireless customers to send voice mes
2002fOO732O7 A1 6/2002 Widger et al. sages from 2-Way devices', Oct. 29, 2001, pp. 1-2.
2002fOO94067 A1 7/2002 August Internet Traveler, “Welcome to the Inter.Net Communicator Tour',
2002fO142756 A1 10/2002 Rutledge et al. http://www.internet/traveler/tour/communicator messaging.php,
2003/0039339 A1 2/2003 Luehrig et al. downloaded Oct. 14, 2003, p. 1.
2003/0041048 A1 2/2003 Balasuriya
2003. O103600 A1 6, 2003 Potter J. Rosenberg, H. Schulzrinne, Internet Draft, “SIP for Presence.”
2003. O105854 A1 6/2003 Thorsteinsson et al. http://www.alternic.org/drafts drafts-r-Sidraft-rosenberg sip-pip
2003/01 12948 A1* 6/2003 Brown et al. ............ 379,207.04 00.txt, Nov. 13, 1998, Bell Laboratories, Columbia, pp. 1-31.
2003/0129.968 A1 7, 2003 Earl Joseph, Anthony D. et al., “The Case for Services over Cascaded
2003/019 1814 A1* 10, 2003 Tran .............................. 709/217 Networks'. EECS Department, CS Division, University of Califor
2003/0232629 A1 12/2003 Jang et al. nia, Berkeley, http://iceberg.cs.berkeley.edu/. International Confer
2004.0024882 A1 2, 2004 Austin et al. ence on Wireless and Mobile Multimedia 1998, pp. 1-9.
2004/0072585 A1 4/2004 Le et al.
2004/O122979 A1 6, 2004 Kirkland MobileShop, “SMS also know as text messaging'. http://www.
2004/0203919 A1 10, 2004 Ross et al. mobileshop.org/howitworks. Sms.htm, downloaded Oct. 14, 2003,
2005/0O27385 A1 2, 2005 Yuch pp. 1-2.
2005/OO37785 A1 2, 2005 Chen Schulzrinne, H. et al., RPID: Rich Presence Extensions to the Pres
2005, 0071253 A1 3/2005 Yang ence Information Data Format (PIDF), draft-ietf-simple-rpid-06.txt,
2005/O107130 A1 5/2005 Peterson, II Jun. 2, 2005, http://www1.ietforg/mail-archive/web/simple current?
2005/O136955 A1 6, 2005 Mumicket al. msg05398.html, downloaded Nov. 15, 2006, pp. 1-35.
2005/019 1994 A1 9/2005 May et al. Schulzrinne, H. et al., RPID: Rich Presence Extensions to the Pres
2005, 0192061 A1 9/2005 May et al. ence Information Data Format (PIDF), draft-ietf-simple-rpid-10.txt,
2005/0273327 A1 12, 2005 Krishnan
2006,0003803 A1 1/2006 Thomas et al. Dec. 20, 2005, pp. 1-41.
2006/0239419 A1 10/2006 Joseph et al. Schulzrinne, H. et al., RPID: Rich Presence Extensions to the Pres
2006/0288099 A1 12/2006 Jefferson et al. ence Information Data Format (PIDF), draft-ietf-simple-rpid-10.txt,
2007/0047522 A1 3/2007 Jefferson et al. Dec. 4, 2005, pp. 1-35.
2011 O151852 A1 6/2011 Olincy et al. Sonim Technologies, Inc., “Integrated voice and text messanging
OTHER PUBLICATIONS over GPRS showcased jointly by Sonim, Symbian and Texas Instru
ments'. Sonim Press Release, Dec. 2, 2002, pp. 1-2.
Second Office Action for CN Patent Application No. 200680027864. Symbian Ltd., “Symbian OS Version 7.0: Functional description'.
9, dated Oct. 25, 2010 (14 pages). Revision 1.5, Feb. 2003, pp. 1-24.
“Company Overview'. http://www.fastmobile.com/company over Symbian Ltd., “Symbian OS Version 7,0s: Functional description'.
view.html, downloaded Nov. 5, 2003, p. 1. Revision 2.1, Jun. 2003, pp. 1-29.
“Introducing the TellmeVoice Application Network”, Tellme, http:// Symbian Ltd., “Technology: Creating Symbian OS phones'. http://
www.tellme.com/products, downloaded Oct. 2, 2003, p. 1. www.symbian.com/technology/create-symb-OS-phones.html,
“iotum History.” Iotum Corp. http://iotum.com/simplyrelevant? downloaded Nov. 5, 2003, p. 1-8.
2006/04/03/iotum-history, downloaded May 15, 2006, pp. 1-4. Symbian Ltd., “Technology: Why is a different operating system
“messaging”, Vodafone Group, 2001, http:www.vodafone.co.nz/ needed'. http://www.symbian.com/technology/why-diff-os.html,
business, 10.2.3 messaging.jsp, downloaded Oct. 14, 2003, pp. 1-2. downloaded Nov. 5, 2003, pp. 1-5.
“Microsoft Windows Messenger: Go Beyond Text with Voice &
Video Chats'. Dell Inc. http://www.dell.com/us/en/dhs/topics/ Verizon Wireless, “TXT messaging”. http://www.vtext.com/cus
segtopic 002 Xp im.htm, downloaded Oct. 2, 2003, pp. 1-2. tomer site/jsp/messaging lo.jsp, downloaded Oct. 2, 2003, p. 1.
“Our Solution.” Iotum Corp. http://www.iotum.com/our solution. W3C, Voice Extensible Markup Language (VoiceXML) Version 2.0,
php, downloaded May 15, 2006, pp. 1-2. W3C, www.w3.org, Feb. 20, 2003.
US 8,744.407 B2
Page 3
(56) References Cited Notice of Rejection for CN Patent Application No. 2006800279649,
dated Jan. 6, 2012 (11 pgs.).
OTHER PUBLICATIONS
Partial English translation of the Notice of Rejection for CN Patent
Yahoo!Messenger, “Yahoo!Messenger Talk for Free!”. http://mes- Application No. 2006800279649, dated Jan. 6, 2012 (10 pgs.).
senger.yahoo.com/messenger/help? voicechat.html, downloaded Oct.
2, 2003, pp. 1-2.
Third Office Action for CN Patent Application No. 2006800279649,
dated Apr. 8, 2011 (11 pages). * cited by examiner
U.S. Patent Jun. 3, 2014 Sheet 1 of 2 US 8,744.407 B2
Mobile phone
Office phone
Home phone
Mobile SMS/pager from mobile phone or PDA
is5 Home/office SMS (to office/home PC)
6 Mobile Online chat (to mobile phone or PDA)
Home Online chat (Net Meeting, AOL, ICO etc.)
8 Voice mail with instant notification to mobile devices of the user
| 9 | Voice mail without notification to mobile devices
10 office fax
Mobile Email (Blackberry etc.)
Reject
User defined
FIGURE 1
Kinship family members, love ones

Relatives and friends
Boss and VEP
Colleagues
Subordinates
Business acquaintances
VIP Clients
Clients
Secretary
User defined
FIGURE 2
U.S. Patent Jun. 3, 2014 Sheet 2 of 2 US 8,744.407 B2
UrgClass Life threatening - interrupt at any time and occasion

UrgClass2 Urgent confirmed meeting reminder-interruption
allowed
UrgClass7 Cold calls from unknown person

UrgClass8 User defined
FIGURE 3
FIGURE 4
UrgClass4-6 MyBusyStatel
MyBusyState2-3 |All
MyBusyStates All
UrgClass7-8 All
FIGURE 5
US 8,744.407 B2
1. 2
SYSTEMS AND PROCESSES TO MANAGE office SMS, mobile online chat, home online chat, a voice
MULTIPLE MODES OF COMMUNICATION mail with/without instant notification, an office fax, a home
fax, a mobile email, and an email.
CROSS-REFERENCE TO RELATED In one embodiment, the user receives the message through
APPLICATIONS a handheld device, such as a cellular phone. In another
embodiment, the message is electronically conveyed based
This application is a continuation of U.S. patent applica on Internet protocol through a website.
tion Ser. No. 1 1/452,115, filed Jun. 12, 2006, now U.S. Pat. In one embodiment, though the process allows the user to
No. 7,729,688, and entitled “SYSTEMS AND PROCESSES receive the message, the person is not aware of the contact
TO MANAGE MULTIPLE MODES OF COMMUNICA 10
information of the user. For example, the person is not aware
TION', which is hereby incorporated herein by reference, of the phone number of the cellular phone that the user used
which application is a continuation-in-part application of to talk to the person. This prevents the person from directly
U.S. patent application Ser. No. 11/006,343, filed Dec. 7, accessing the user without going through an intermediate
2004, now U.S. Pat. No. 7,116,976, and entitled “ADAPT control, such as a website. Similarly, the user does not have to
15
ABLE COMMUNICATION TECHNIQUES FOR ELEC be aware of the contact information of the person.
TRONIC DEVICES.” which is hereby incorporated herein by In another embodiment, the defined access priority of the
reference, which claims priority to U.S. Provisional Patent person is stored at a website, allowing the website to access
Application No. 60/527.565, filed Dec. 8, 2003, entitled
“ADAPTABLE COMMUNICATION TECHNIQUES FOR Such information without asking for the user's permission. In
ELECTRONIC DEVICES.” and which is hereby incorpo one embodiment, the defined access priority is stored in a
rated herein by reference. private database under the user's control.
This application, by way of U.S. patent application Ser. No. Other aspects and advantages of the present invention will
1 1/452,115, also claims priority to U.S. Provisional Patent become apparent from the following detailed description,
Application No. 60/689,686, filed Jun. 10, 2005, entitled which, when taken in conjunction with the accompanying
SYSTEMS AND PROCESSES TO MANAGE MULTIPLE 25 drawings, illustrates by way of example the principles of the
MODES OF COMMUNICATION,” and which is hereby invention.
incorporated herein by reference.
BRIEF DESCRIPTION OF THE DRAWINGS
BACKGROUND OF THE INVENTION
30 FIG. 1 shows a number of intelligent communication
For many years, other than mails from post offices, we modes according to one embodiment of the invention.
typically only received information from afar through tele FIG. 2 shows a number of contact classes according to one
phones. However, in the past few years, ways that others can embodiment of the invention.
send us information have increased significantly. Just to list a FIG.3 shows a number of urgency classes according to one
few different modes of communication, we can be reached 35 embodiment of the invention.
from standard deskphones, fax, cellphones, electronic mails, FIG. 4 shows a number of statuses of a user according to
and instant messages. In addition, we can have more than one one embodiment of the invention.
phone number and multiple electronic mail addresses. There FIG. 5 shows one embodiment of an example of an Access
are people we like to communicate with, and there are those Priority Database according to one embodiment of the inven
we prefer to avoid. Managing information from all such dif 40 tion.
ferent modes can be quite time consuming. Same numerals in FIGS. 1-5 are assigned to similar ele
It should be apparent from the foregoing that there is still a ments in all the figures. Embodiments of the invention are
need to help manage the numerous modes of communication. discussed below with reference to FIGS. 1-5. However, those
skilled in the art will readily appreciate that the detailed
SUMMARY OF THE INVENTION 45 description given herein with respect to these figures is for
explanatory purposes as the invention extends beyond these
Different embodiments of a computer-implemented sys limited embodiments.
tem and method to manage the communication of a user are
disclosed. A person tries to electronically convey a message DETAILED DESCRIPTION OF THE INVENTION
to the user. In one embodiment, the status of the user is 50
identified; the identity of the person is identified; the urgency One embodiment of the invention can automatically
of the message is identified; the access priority of the person remove unwanted communications. Certain communications
is determined based on the person's identity; and a process is are relatively easy to determine to be unwanted. Such as
set to manage the message using one or more rules, and in marketing cold calls and wrong number calls. Other commu
view of the status of the user, the access priority of the person 55 nications may be more difficult. They can depend not just on
and the urgency of the message. the Sources of the communication, but also the conditions or
Based on different embodiments, the status of the user status of the receiver (a user) of the communication. The
depends on the current activity or location of the user, or the status can be related to the user's current activity and/or
current time. The status of the user can also be defined by the location. For example, when the user is on a train going to
user. Similarly, the access priority of the person can be 60 work, the user probably does not mind chatting with his
defined by the user, or is set depending on the user's reaction grandchild. However, if the user is having his yearly review
towards a prior message from the person. Also, the urgency of meeting with his boss, the user probably would prefer to avoid
the message is set by the person. the call from his grandchild, unless it is an emergency. Based
The process can depend on the mode of communication of on the embodiment, communications from Sources the user
the message. For example, the mode of communication can 65 wants to postpone receiving can be automatically diverted.
include a mobile phone, an office phone, a home phone, a In one embodiment, the user can get appropriate notifica
mobile SMS, a pager from a mobile phone or PDA, a home/ tion on the source of the incoming communication request.
US 8,744.407 B2
3 4
The attributes of the notification can depend on the urgency of In one embodiment, the portal can dynamically change the
the communication and/or the status of the user. access priorities of a caller trying to reach the user. For
The user may receive information from different modes of example, previously the caller is of high priority to the user,
communication. For example, the user can have mobile and the user has set her access priorities accordingly. Lately,
phones, fixed lines at home or office, emails, SMS, and faxes, every time the caller trying to reach the user, the request was
with their different numbers and/or addresses. One embodi denied. After a preset number of rejections, the portal can
ment can help the user efficiently manage information from automatically send a message to the user, asking the user if the
the different modes. The user only has to remember one user would like to lower the access priority of the caller. If the
specific address from one mode of communication. Through response is affirmative, the caller's priority is automatically
that address, the user can receive communications from all 10 reduced.
modes of communication, independent of where the user is, In another embodiment, the user does not have to set pri
or the type of hardware the user has. This allows the user to orities of each contact. The system monitors every call, and
efficiently maintain his communication from the numerous provides the contacts identity to the user. Based on the user's
modes even when he is traveling. For example, the user does reaction to the call (e.g. accepting or rejecting it), the system
not have to change phones (and the phone numbers) when he 15 automatically sets the contacts priorities. In one embodi
moves from areas covering 3G to areas that do not. ment, the system can then query the user for approval on the
A number of embodiments depend on the different modes setting, and allow the user to adjust it as necessary. In another
of communication converging onto the internet protocol plat embodiment, the system can continue to modify the caller's
form. A communication gateway or a portal is formed allow priorities based on the user's reaction to the caller's subse
ing the user to receive communications from numerous quent calls.
Sources through different modes. This, in turn, could reduce In one embodiment, the user could keep information he
the numerous addresses the user has to remember, to one believes to be sensitive local in a different database. Such
address. For example, an e-mail address for the user can serve information can be stored securely under the user's direct
as an access identifier for the different communication control. The portal can retrieve information from the different
addresses from different communication modes. The access 25 database when required. In another embodiment, the user can
identifier can become the user's digital identity. In one restrict or limit such retrieval process.
embodiment, the user's other types of identification, such as Additional confidentiality can be provided. In one embodi
the user's driver licenser number, can be the user's access ment, using phone calls as an example, the user can be aware
identifier. of the identity of the caller even without being informed of the
One embodiment of the invention uses an open portal 30 number of the caller. Similarly, the caller can reach the user
based on the web. Based on the portal, the user can securely without being aware of the number of the phone the user is
determine who can reach him at what conditions. This can be using to receive the call. The user can keep his location and/or
done based on a status indicator. As an example, this indicator status confidential but still can receive the communication.
is determined according to the status of the user, the access This can be useful because there are situations, for example,
priorities of the person trying to reach the user (or the rela 35 when the user does not want to disclose his contact informa
tionship or the lack of relationship between the user and the tion but the user needs to receive services provided by the
person), and/or the urgency of the message from the person. caller.
The status of the user can be dynamically determined, based One approach to maintain such confidentiality while main
on the current condition(s) of the user. The portal can allow taining real-time communication is based on a system that
the user and the person to select different options, which can 40 digitally identifies the identities of the caller and the receiver.
be modified as desired. For example, the relationship can be Note that the term caller is used in general. It is not just limited
preset by the user and stored in a database, while the urgency to phone calls, but they can be any person or entity requesting
of the message can be set by the person. to communicate with the user, Such as trying to send a mes
Thus, in one embodiment, the portal can be used to control sage to the user. As a separate note, the caller can also be a user
the selection and setting of different intelligent communica 45 of different embodiments of the invention.
tion modes for the user. These intelligent communication After determining the identities, the system can establish
modes allow priorities of various kinds of communication connections between the caller and the user in real time.
options to be set by the user. The portal allows worldwide Though contacts are established, the system only needs to
access to the user, and can dynamically determine, for ensure the identities of the caller and the user to each other.
example, whether a call initiated at different time by different 50 However, the system does not have to disclose the phone
callers should be accepted by the user in real-time or handled numbers, electronic addresses, physical locations and/or
by other mechanisms. From this information, communication other attributes of the caller and the user to each other. In one
requests can be classified, for example, into different degrees embodiment, real time implies that the time required for the
of undesirability. Some requests can be automatically identification is similar to the typical time required to set up,
blocked from the user. Others can be diverted and handled by 55 for example, a telephone call. The system can be a portal
other mechanism, such as divertingaphone call to an email or based on the web.
Voice mail. In one embodiment, a portal also holds the user's electronic
In one embodiment, the portal or gateway also includes a calendar. The calendar can be programmable, with entries set
database to keep track of the user's different contacts or by the user. The portal can automatically and securely set
acquaintances, and the access priorities of each contact. The 60 appointments for the user since the portal knows the identity
user can modify information in the database. Such as assign of the caller, and the status and schedule of the user. For
ing and/or changing the priorities of the contacts. Based on example, the appointment can be for a conference call.
the information (or lack of information) in the database of the To illustrate, in one embodiment, a portal provides a num
contact trying to access the user, and based on the status of the ber of intelligent communication modes (ICM) for the user to
user, the gateway can automatically select an intelligent mode 65 select as shown in FIG.1. There are three columns in the table.
of communication for the user. This selection can be done If the communication mode selected in the second column
dynamically. does not work, the portal automatically defaults to the corre
US 8,744.407 B2
5 6
sponding approach in the third column. For example, under web server based on Internet protocol. A portal or gateway
ICM 1, if the mobile phone is busy, default to voice mail. approach could provide general Internet access to one or more
Some of the selections do not have any default because it may embodiments of the communication management systems so
not be necessary to default. For example, under ICM 8, the that users can configure the system behavior they desire. The
incoming message goes directly to Voice mail with instant portal or gateway can then facilitate download of a database
notification to mobile devices of the user. The incoming mes or update thereto to a communication device, such as a phone.
sage can usually go to Voice mail. There is no need to default. Also, as explained above, based on an embodiment, a user
As a receiver of communication, the user can define a could efficiently maintain his communication, and does not
number of contact classes, as shown in FIG. 2. The user can even have to change phones when he moves from areas cov
set up a number of urgency classes, as shown in FIG. 3. The 10 ering 3G to areas that do not. These phones could be based on
user can define a number of status, as shown in FIG. 4. Then, different communication mechanisms, such as GSM,
based on tables in FIGS. 1-4, the user can set up an Access CDMA 3G and 4G systems. Also as explained above, the
Priority Database for different ContactClasses, as shown in user could keep information in local databases, such as in
FIG.S.
Such a phone. For example, the intelligent communication
As another example, the user can categorize the following
15 modes shown in FIG. 1 for the user to select are in the phone.
The user could define the contact classes, such as the ones
contacts into the corresponding ContactClasses: shown in FIG. 2; set up the urgency classes, such as the ones
shown in FIG. 3; define the statuses, such as the ones shown
in FIG. 4; set up the Access Priority Database, such as the one
Alice (Wife) ContactClass1 shown in FIG. 5; and categorize a number of the user's con
Peter (Close Friend) ContactClass2 tacts into the corresponding ContactClasses, all in the phone.
Colina (Close Friend's ContactClass2 When a caller places a call to the phone, based on information
wife) previously set in the phone and based on the urgency class
selected by the caller, the phone could automatically manage
Peter wants to make a mobile phone call to the user. In one 25 the communication. Note that the phone does not have to be a
embodiment, Peter calls a portal. As an example, the portal cellular phone. In one embodiment, the phone is a desktop
can be the user's ISP. The portal first verifies the caller's phone.
identity to be Peter. This can be done, for example, by a public Again as explained above, the person or the caller trying to
key challenge based on Peter having a public key digital contact the user could select different options. For example,
certificate. In another example, Peter is also a registered user 30 the urgency of the message can be set by the caller. This
of the portal. Then, Peter's identity can be more readily iden selection is typically in the call setup phase. In one embodi
tified or verified. ment, the caller has pre-selected the urgency class before
In one embodiment, after verification, a virtual address/ making the call. In another embodiment, if the caller has not
number for the communication session is created allowing selected the urgency class, the system could prompt the caller
Peter to reach the user, which can be by phone. The user's 35 to input an urgency class or status before the call or message
phone number does not have to be disclosed to Peter. Simi is routed to the user. In yet another embodiment, different
larly, Peter's mobile phone number does not have to be dis urgency classes could be defined by the caller.
closed to the user. The portal can assure the user that the Further, the computer-implemented methods and systems
person calling is Peter based on an identification verification discussed above can be used in conjunction with one or more
process, such as ones described above. 40 of the various approaches discussed in U.S. patent application
In establishing contact, the portal can access the user's Ser. No. 11/006,343. For example, the automated actions or
database and determine that Peter belongs to ContactClass2. decisions (e.g., intelligent secretary, decision 204 in FIG. 2,
The database can, for example, be in the portal. etc.) of U.S. patent application Ser. No. 11/006,343 can be
In another embodiment, the database is in a personal com automatically made by the systems/methods described above.
munication device of the user. The portal accesses the per 45 Still further, the various approaches discussed in U.S. patent
sonal communication device to determine Peter's Contact application Ser. No. 11/006,343 can be used in conjunction
Class. with one or more the various methods/systems discussed
Based on the ContactClass information, the status of the above. For example, the systems/methods described above
user and Peter's urgency setting, the user may receive Peter's can use the messaging approaches (e.g., audio or textual
call directly. As another example, Peter may be asked to leave 50 messages) described in U.S. patent application Ser. No.
a voice mail to the user, while the user is notified by a mobile 11/006,343.
short message regarding an incoming call from Peter. Other embodiments of the invention will be apparent to
As additional examples, in one embodiment, location those skilled in the art from a consideration of this specifica
information of the user could be determined based on GPS tion or practice of the invention disclosed herein. It is
information from, for example, the user's cell phone. 55 intended that the specification and examples be considered as
In one embodiment, the user receives messages through a exemplary only, with the true scope and spirit of the invention
handheld device. Such as a phone, and the phone has a Switch. being indicated by the following claims.
The Switch can be a physical button or a software setting. Such
as a pull-down menu. The user could set his status dynami What is claimed is:
cally by changing the physical or logical position of the 60 1. A non-transitory computer-implemented method to
Switch. For example, one position can indicate that the user is manage the communication of a user via a server based on a
very busy, and should only be interrupted by an urgent mes communication protocol, in view of a person, using a first
sage from the user's closest contacts, such as his wife or device, trying to electronically convey a message from the
parents. Another position can indicate that the user's status first device to a second device of the user, the method com
allows the user to receive any messages from anyone. 65 prising:
As explained above, based on an embodiment, a message is receiving, by the server, the message from the person using
electronically conveyed by a central network server, such as a the first device;
US 8,744.407 B2
7 8
identifying, by the server, the person attempting to elec wherein the at least one computing device is configured to
tronically convey the message; and receive the message from the person using the first
setting, by the server, a process for the message using one device;
or more rules based on at least a status associated with identify the person attempting to electronically convey
the user and an access priority associated with the per the message; and
son, the access priority depending on the person, set a process for the message using one or more rules
wherein the server is configured to have access to contact based on at least a status associated with the user and
information of the person, an access priority associated with the person, the sta
wherein even if the process includes transmitting the mes tus depending at least in part on the current activity or
Sage to the second device via contact information of the 10
location of the user, or the current time, and the access
user, the server does not provide the contact information priority depending on the person,
of the user to the first device to inhibit the person from wherein the server is configured to have access to contact
sending messages to the user without via the server, and information of the person,
wherein the access priority associated with the person is
configured to be set by the server depending at least in 15 wherein even if the process includes transmitting the mes
part on the user reacting by accepting or not accepting a Sage to the second device via contact information of the
prior message from the person. user, the server does not provide the contact information
2. The non-transitory computer-implemented method as of the user to the first device to inhibit the person from
recited in claim 1, wherein the access priority associated with sending messages to the user without via the server,
the person is allowed to be set by the user. wherein the server can be restricted from accessing the
3. The non-transitory computer-implemented method as status of the user from the second device, and
recited in claim 1, wherein the process for the message also wherein the server can be restricted from accessing the
depends on an urgency level of the message, which is allowed access priority of the person from the second device.
to be set by the person. 9. A server as recited in claim 8, wherein the process for the
4. The non-transitory computer-implemented method as 25 message also depends on an urgency level of the message,
recited in claim 3, which is allowed to be set by the person.
wherein the status associated with the user and the access 10. A server as recited in claim 8,
priority associated with the person are stored at the sec wherein the process for the message also depends a mode
ond device, of communication of the message, and
wherein the server can be restricted from accessing the 30 wherein the mode of communication is selected from the
status associated with the user from the second device, list of a mobile phone, an office phone, a home phone, a
and mobile SMS from a mobile phone or PDA, a pager from
wherein the server can be restricted from accessing the a mobile phone or PDA, a home/office SMS, mobile
access priority associated with the person from the sec online chat, home online chat, a Voice mail with/without
ond device. 35 instant notification, an office fax, a home fax, a mobile
5. The non-transitory computer-implemented method as email, and an email.
recited in claim 1, 11. A server as recited in claim 8, wherein the server is
wherein the method depends on a mode of communication configured to keep an electronic calendar of the user, and to
of the message, and set an appointment for the user with the person in view of
wherein the mode of communication is selected from the 40 information in the calendar.
list of a mobile phone, an office phone, a home phone, a 12. A server as recited in claim 8, wherein access priority
mobile SMS from a mobile phone or PDA, a pager from associated with the person to help set the process depends at
a mobile phone or PDA, a home/office SMS, mobile least in part on a reaction of the user towards a prior message
online chat, home online chat, a Voice mail with/without from the person.
instant notification, an office fax, a home fax, a mobile 45 13. A server as recited in claim 8, wherein at least one
email, and an email. computing device is configured to ask the user regarding
6. The non-transitory computer-implemented method as changing access priority associated with the person in view of
recited in claim 1, at least a reaction of the user towards a prior message from the
wherein the server keeps an electronic calendar of the user, person, and to change access priority associated with the
and 50 person if so instructed by the response of the user.
wherein an appointment is set by the server for the user 14. A server as recited in claim 8, wherein the status asso
with the person in view of information in the calendar. ciated with the user is allowed to be selected by the user from
7. A non-transitory computer-implemented method as a plurality of preset statuses.
recited in claim 1, wherein even if the process includes trans 15. A server as recited in claim 14, wherein the process
mitting the message to the second device via contact infor 55 includes asking the person to leave a Voice mail for the user in
mation of the user, the server does not provide the contact view of an attribute regarding the message.
information of the person to the second device to inhibit the 16. A server as recited in claim 14, wherein the process
user from sending messages to the person without via the includes sending a text message regarding the message to the
SeVe. USC.
8. A server based on a communication protocol for man 60 17. A server as recited in claim 14, wherein the process
aging the communication of a user, in view of a person, using includes responding to the message by retrieving an elec
a first device, trying to electronically convey a message from tronic mail to send to the person in view of an attribute
the first device to a second device of the user, the second regarding the message.
device being a handheld communication device, the server 18. A server as recited in claim 14,
comprising: 65 wherein the message can be voice or text, and
at least one computing device; and wherein the communication protocol is the Internet proto
at least one storage device, col.
US 8,744.407 B2
10
19. A server as recited in claim 8, wherein the mode of communication is selected from the
wherein at least one computing device is further configured list of a mobile phone, an office phone, a home phone, a
to identify the person to the user, and mobile SMS from a mobile phone or PDA, a pager from
wherein the process includes a mobile phone or PDA, a home/office SMS, mobile
(i) determining whether the message should be delivered online chat, home online chat, a Voice mail with/without
to the user based on at least one attribute of the mes instant notification, an office fax, a home fax, a mobile
Sage, email, and an email.
(ii) delivering the message to the user if it is determined 27. A non-transitory computer readable medium as recited
that the message should be delivered; and 10
in claim 24, wherein said computer readable medium further
(iii) declining to deliver the message to the user if it is comprises computer program code to keep an electronic cal
determined that the message should not be delivered. endar of the user, and to set an appointment for the user with
20. A server as recited in claim 19, wherein the message is the person in view of information in the calendar.
a text message. 28. A non-transitory computer readable medium as recited
21. A server as recited in claim 19, wherein at least one 15 in claim 24, wherein access priority associated with the per
computing device is further configured to ask the user regard son to help set the process depends at least in part on a reaction
ing access priority associated with the person, and set access of the user towards a prior message from the person.
priority associated with the person in view of a response from 29. A non-transitory computer readable medium as recited
the user. in claim 24, wherein the computer readable medium further
22. A server as recited in claim 8, wherein the process comprises:
includes setting an appointment between the user and the computer program code for permitting the user to change
person in view of a schedule of the user. access priority associated with the person; and
23. A server as recited in claim 8, wherein even if the computer program code for changing access priority asso
process includes transmitting the message to the second ciated with the person if so instructed by the user.
device via contact information of the user, the server does not 25 30. A non-transitory computer readable medium as recited
provide the contact information of the person to the second in claim 24,
device to inhibit the user from sending messages to the person wherein the computer readable medium further comprises
without via the server. computer program code to identify the person to the
24. A non-transitory computer readable medium including user,
at least executable computer program code tangibly stored 30
wherein the process includes not delivering the message to
therein for manage the communication of a user via a server the user in view of an attribute regarding the message,
based on a communication protocol, in view of a person, and
using a first device, trying to electronically convey a message wherein the status associated with the user is allowed to be
from the first device to a second device of the user, the second selected by the user from a plurality of preset statuses.
device being a handheld communication device, said com 35
31. A non-transitory computer readable medium as recited
puter readable medium comprising:
computer program code for receiving, by the server, the in claim 30, wherein the process includes asking the person to
message from the person using the first device; leave a Voice mail for the user in view of an attribute regarding
computer program code for identifying the person attempt the message.
ing to electronically convey the message; and 40 32. A non-transitory computer readable medium as recited
computer program code for setting a process for the mes in claim 30, wherein the computer readable medium further
Sage using one or more rules based on at least a status comprises computer program code to ask the user regarding
associated with the user, and an access priority associ access priority associated with the person, and set access
ated with the person, the status depending at least in part priority associated with the person in view of a response from
on the current activity or location of the user, or the 45 the user.
current time, and the access priority depending on the 33. A non-transitory computer readable medium as recited
person, in claim 30,
wherein the server is configured to have access to contact wherein the message can be voice or text, and
information of the person, wherein the communication protocol is the Internet proto
wherein even if the process includes transmitting the mes 50 col.
Sage to the second device via contact information of the 34. A non-transitory computer readable medium as recited
user, the computer program code does not provide the in claim 24, wherein the process includes sending a text
contact information of the user to the first device to message regarding the message to the user.
inhibit the person from sending messages to the user 35. A non-transitory computer readable medium as recited
without via the server, 55 in claim 24, wherein the message is a text message.
wherein the server can be restricted from accessing the 36. A non-transitory computer readable medium as recited
status of the user from the second device, and in claim 24, wherein the process includes responding to the
wherein the server can be restricted from accessing the message by retrieving an electronic mail to send to the person
access priority of the person from the second device. in view of an attribute regarding the message.
25. A non-transitory computer readable medium as recited 60 37. A non-transitory computer readable medium as recited
in claim 24, wherein the process for the message also depends in claim 24, wherein the process includes setting an appoint
on an urgency level of the message, which is allowed to be set ment between the user and the person in view of a schedule of
by the person. the user.
26. A non-transitory computer readable medium as recited 38. A non-transitory computer readable medium as recited
in claim 24, 65 in claim 24, wherein even if the process includes transmitting
wherein the process for the message also depends a mode the message to the second device via contact information of
of communication of the message, and the user, the computer program code does not provide the
US 8,744.407 B2
11 12
contact information of the person to the second device to 48. A non-transitory computer-implemented method as
inhibit the user from sending messages to the person without recited in claim 43, wherein the process includes responding
via the server. to the message by retrieving an electronic mail to send to the
39. A non-transitory computer-implemented method to person in view of an attribute regarding the message.
manage the communication of a user via a server based on a 49. A non-transitory computer-implemented method as
communication protocol, in view of a person, using a first recited in claim 43, wherein the method further comprises
device, trying to electronically convey a message from the asking the user regarding the access priority associated with
first device to a second device of the user, the second device the person, and setting access priority associated with the
being a handheld communication device, the method com person in view of a response from the user.
prising: 10 50. A non-transitory computer-implemented method as
receiving, by the server, the message from the person using recited in claim 43,
the first device; wherein the message can be voice or text, and
identifying the person attempting to send the message to wherein the communication protocol is the Internet proto
the user, and col.
setting a process regarding delivering the message to the 15 51. A non-transitory computer-implemented method as
user using one or more rules based on at least a status recited in claim 39, wherein the process includes setting an
associated with the user and an access priority associ appointment between the user and the person in view of a
ated with the person, the status depending at least in part schedule of the user.
on the current activity or location of the user, or the 52. A non-transitory computer-implemented method as
current time, and access priority depending on the per recited in claim 39, wherein even if the message is delivered
SOn, to the second device via contact information of the user, the
wherein the server is configured to have access to contact server does not provide the contact information of the person
information of the person, to the second device to inhibit the user from sending messages
wherein even if the message is delivered to the second to the person without using the server.
device via contact information of the user, the server 25 53. A server based on a communication protocol for man
does not provide the contact information of the user to aging the communication of a user, in view of a person, using
the first device to inhibit the person from sending mes a first device, trying to electronically convey a message from
Sages to the user without using the server, the first device to a second device of the user, the second
wherein the server can be restricted from accessing the device being a handheld communication device, the server
status of the user from the second device, and 30 comprising:
wherein the server can be restricted from accessing access at least one computing device; and
priority of the person from the second device. at least one storage device,
40. A non-transitory computer-implemented method as wherein at least one computing device is configured to
recited in claim 39, wherein the process for the message also receive the message from the person using the first
depends on an urgency level of the message, which is allowed 35 device;
to be set by the person. identify the person attempting to electronically convey
41. A non-transitory computer-implemented method as the message; and
recited in claim 39, wherein access priority associated with set a process for the message using one or more rules
the person to help set the process depends at least in part on a based on at least a status associated with the user and
reaction of the user towards a prior message from the person. 40 an access priority associated with the person, the sta
42. A non-transitory computer-implemented method as tus depending at least in part on the current activity or
recited in claim 39, wherein the method further comprises location of the user, or the current time, and access
asking the user regarding changing access priority associated priority depending on the person,
with the person in view of at least a reaction of the user wherein the server is configured to have access to contact
towards a prior message from the person, and to change 45 information of the person,
access priority associated with the person if so instructed by wherein even if the process includes transmitting the mes
the response of the user. Sage to the second device via contact information of the
43. A non-transitory computer-implemented method as user, the server does not provide the contact information
recited in claim 39, of the user to the first device to inhibit the person from
wherein the status associated with the user is allowed to be 50 sending messages to the user without via the server,
selected by the user from a plurality of preset statuses, wherein the server is configured to be aware of availability
and of the user, and
wherein the method further comprises identifying the per wherein the server is configured to send information to the
son to the user. person regarding availability of the user.
44. A non-transitory computer-implemented method as 55 54. A server as recited in claim 53, wherein based on the
recited in claim 43, wherein the process includes not deliver identity of the person, the server is configured to send infor
ing the message to the user in view of an attribute regarding mation to the person regarding availability of the user.
the message. 55. A server as recited in claim 53, wherein the process for
45. A non-transitory computer-implemented method as the message also depends on an urgency level of the message,
recited in claim 43, wherein the process includes asking the 60 which is allowed to be set by the person.
person to leave a voice mail for the user in view of an attribute 56. A server as recited in claim 53,
regarding the message. wherein the process for the message also depends a mode
46. A non-transitory computer-implemented method as of communication, and
recited in claim 43, wherein the process includes sending a wherein the mode of communication is selected from the
text message regarding the message to the user. 65 list of a mobile phone, an office phone, a home phone, a
47. A non-transitory computer-implemented method as mobile SMS from a mobile phone, a pager from a mobile
recited in claim 43, wherein the message is a text message. phone, a home/office SMS, mobile online chat, home
US 8,744.407 B2
13
online chat, a Voice mail with/without instant notifica
tion, an office fax, a home fax, a mobile email, and an
email.
57. A server as recited in claim 53, wherein the server is
configured to keep an electronic calendar of the user, and to 5
set an appointment for the user with the person in view of
information in the calendar.
configured to set the access priority associated with the per
son depending at least in part on a reaction of the user to a 10
prior message from the person.
configured to allow the user to set access priority associated
with the person.
15
UNITED STATES PATENT AND TRADEMARK OFFICE

CERTIFICATE OF CORRECTION
PATENT NO. : 8,744.407 B2 Page 1 of 3
APPLICATIONNO. : 12/798.995
DATED : June 3, 2014
INVENTOR(S) : Cheung et al.
It is certified that error appears in the above-identified patent and that said Letters Patent is hereby corrected as shown below:
On Title Page 2, Item (56) Other Publications:

“Internet Traveler, “Welcome to the Inter.Net Communicator Tour,
http://www.internet/traveler/tour/communicator messaging.php, downloaded Oct. 14, 2003, p. 1.
should be --Internet Traveler, “Welcome to the Inter.Net Communicator Tour,
http://www.inter.net/traveler/tour/communicator messaging.php, downloaded Oct. 14, 2003, p. 1.--.
In the Claims
Column 8, line 41 (claim 12, line 1) “A server as recited in claim 8, wherein access priority should be
-A server as recited in claim 8, where the access priority--.
Column 8, lines 45-50 (claim 13, lines 1-6) “A server as recited in claim 8, wherein at least one
computing device is configured to ask the user regarding changing acceSS priority associated with the
person in view of at least a reaction of the user towards a prior message from the person, and to
change access priority associated with the person if so instructed by the response of the user.
should be --A server as recited in claim 8, wherein the at least one computing device is configured to
ask the user regarding changing the access priority associated with the person in View of at least a
reaction of the user towards a prior message from the person, and to change the access priority
associated with the person if so instructed by the response of the user.--.
computing device is further configured
should be --A server as recited in claim 8, wherein the at least one computing device is further
configured--.
computing device is further configured to ask the user regarding access priority associated with the
person, and Set access priority
Signed and Sealed this

Seventh Day of April, 2015
74-4-04- 2% 4 Michelle K. Lee
Director of the United States Patent and Trademark Office
CERTIFICATE OF CORRECTION (continued) Page 2 of 3

U.S. Pat. No. 8,744.407 B2
should be --A server as recited in claim 19, wherein the at least one computing device is further
configured to ask the user regarding the access priority associated with the person, and set the acceSS
priority--.
Column 10, lines 13-14 (claim 28, lines 1-2) “computer readable medium as recited in claim24,
wherein access priority
should be --computer readable medium as recited in claim 24, wherein the access priority--.
Column 10, lines 21-23 (claim 29, lines 4-6) “permitting the user to change access priority associated
With the person; and computer program code for changing access priority
should be --permitting the user to change the access priority associated with the person; and computer
program code for changing the access priority--.
Column 10, lines 42-44 (claim 32, lines 3-5) “program code to ask the user regarding access priority
associated with the person, and set access priority
should be -program code to ask the user regarding the access priority associated with the person, and
Set the access priority--.
Column 11, line 20-21 (claim 39, lines 17-18) “and access priority depending on the person
should be --and the access priority depending on the person--.
Column 11, lines 31-32 (claim 39, lines 28-29) “wherein the server can be restricted from accessing
access priority of the person from the second device.”
should be --wherein the server can be restricted from accessing the access priority of the person from
the Second device.--.
Column 11, lines 38-39 (claim 41, lines 2-3) “wherein access priority associated with the person
should be --wherein the access priority associated with the person--.
Column 11, lines 43-47 (claim 42, lines 3-7) “regarding changing access priority associated with the
person in view of at least a reaction of the user towards a prior message from the person, and to
change access priority associated with the person if so instructed by the response of the user.
should be --regarding changing the access priority associated with the person in view of at least a
reaction of the user towards a prior message from the person, and to change the access priority
associated with the person if so instructed by the response of the user.--.
Column 12, lines 8-9 (claim 49, lines 4-5) “setting access priority associated with the person
should be -setting the access priority associated with the person--.
Column 12, lines 33-34 (claim 53, lines 9-10) “wherein at least one computing device is configured to
receive
should be --wherein the at least one computing device is configured to receive--.
Column 12, lines 42-43 (claim 53, lines 18-19) “and access priority depending on the person
should be --and the access priority depending on the person--.
CERTIFICATE OF CORRECTION (continued) Page 3 of 3

U.S. Pat. No. 8,744.407 B2
Column 12, lines 51-54 (claim 53, lines 27-30) “wherein the server is configured to be aware of
availability of the user, and wherein the server is configured to send information to the person
regarding availability of the user.
should be --wherein the server is configured to be aware of the availability of the user, and wherein the
server is configured to send information to the person regarding the availability of the user.--.
Column 12, lines 56-57 (claim 54, lines 2-3) “to send information to the person regarding availability
of the user.
should be --to send information to the person regarding the availability of the user.--.
Column 13, lines 13-14 (claim 59, lines 2-3) “configured to allow the user to set access priority
associated with the person.
should be --configured to allow the user to set the access priority associated with the person.--.
EXHIBIT E
Preliminary comparison of Fortnite with U.S. Patent No. 10,708,727
1. A computer-implemented As shown below, Fortnite utilizes a computer-implemented method to facilitate

method to facilitate electronic electronic communication of a plurality of users using at least a network-based portal at
communication of a plurality of least based on Internet protocol.
users using at least a network-
based portal at least based on
Internet protocol, the method
comprising:
providing a plurality of modes Fortnite provides a plurality of modes of communication to a first user to allow the first
of communication to a first user user to use one of the plurality of modes of communication as a selected mode of
to allow the first user to use one communication for a first message to be sent from the first user to a second user, based
of the plurality of modes of on an identifier associated with the first user previously set by the first user via the
communication as a selected network-based portal.
mode of communication for a
first message to be sent from For example, PC users will be directed to the registration screen shown below. So,
the first user to a second user, players must be registered with their ID (DISPLAY NAME) to use the game (and,
based on an identifier therefore, to send messages in any mode available within the game):
associated with the first user
previously set by the first user
via the network-based portal,
1
https://www.epicgames.com/id/register/epic
“Voice Chat - ON or OFF
NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
you are playing on may have additional communication features that must be restricted
2
separately. Information on platform-specific controls can be found below or by

clicking the “More Settings” button in-game.
• ON: You can hear your teammates and talk to them using a microphone.
• OFF: You cannot hear or talk to teammates.
Text Chat - ON or OFF
NOTE: This setting only enables/disables Fortnite’s in-game text chat. The platform
• ON: You can send and receive text chat messages with your teammates.
• OFF: You cannot send or receive text chat messages with your teammates.”
https://www.epicgames.com/fortnite/en-US/parental-controls
The selected mode of communication can be voice chat.
wherein the plurality of modes The plurality of modes of communication supported by the network-based portal
of communication supported by utilized by Fortnite include at least text communication using a personal computer and
the network-based portal voice communication using a personal computer.
include at least text
communication using a For example, the parental controls shown below, which show both voice and text chat
personal computer, voice options, are available on all platforms, including PC:
communication using a
personal computer, and “Voice Chat - ON or OFF
communication with at least an
image, and NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
3

“Where can I download Battle Royale?

You can download Battle Royale in the following places:
PC/Mac - Fortnite.com (Apple has terminated our ability to develop Fortnite for Mac.
As a result, Fortnite on Mac remains on version 13.40 for Battle Royale/Creative.)
PlayStation - PlayStation Store
Xbox - Xbox Marketplace
Nintendo Switch - Nintendo eShop
Android - Currently available through the Epic Games App on the Samsung Galaxy
Store or epicgames.com. Google has blocked your access to Fortnite on Google Play.
iOS - Apple has blocked your access to Fortnite on iOS devices.”
https://www.epicgames.com/fortnite/en-US/faq
4
“On Mac and PC, anyone on mobile, PS4 or Xbox One who’s in a cross-platform party
can appear in your match — unless you’re playing in Solo mode, where you'll only
play against other Mac and PC players.”
https://www.polygon.com/2018/3/23/17146848/cross-platform-crossplay-ps4-xbox-pc-
switch-ios-mobile-enable-friends-compatible-matchmaking
“Here’s how to chat cross-platform and talk cross-platform with players on PC, PS4,
Xbox One, and mobile.”
https://twinfinite.net/2018/09/fortnite-cross-platform-chat/
The plurality of modes of communication supported by the network-based portal

utilized by Fortnite also includes communication with at least an image.
“The Fortnite ping system calls out anything in the world with an audible (wait for
it) ping, as well as an identifying icon and distance visible to other people on your
team. There's no voice acting involved but it still lets you highlight things like chests,
ammo, weapons and so on, as well as threats, in a way that squad members can easily
identify.
In an ideal world it should help squads of randoms coordinate things better without
having to use headsets and talk to each other. Whether that actually happens or not is
another thing.
The Fortnite ping system lives on the left D-pad button. When you press that it'll ping
whatever you're looking at with a marker and distance. If you're not looking at a
weapon or any sort of item then it'll mark the location on map under your crosshairs. A
double tap on the other hand will indicate danger - a useful way to point out rival
squads.”
https://www.gamesradar.com/fortnite-ping-system-how-to/
wherein messages are eligible In Fortnite, messages are eligible to be received by the second user via the network-
to be received by the second based portal, based on any of the plurality of modes of communication, all depending
user via the network-based on an identifier associated with the second user previously set by the second user via
5
portal, based on any of the the network-based portal, which allows the second user to efficiently maintain the
plurality of modes of second user's communication using the plurality of modes of communication.
communication, all depending
on an identifier associated with For example, players must be registered with their ID to use the game (and, therefore,
the second user previously set to receive messages in any mode available within the game):
by the second user via the
network-based portal, which
allows the second user to
efficiently maintain the second
user's communication using the
plurality of modes of
communication;
6
enabling the second user to Fortnite enables the second user to block the first user from using at least the selected
block the first user from using mode of communication to communicate with the second user via the network-based
at least the selected mode of portal, based on the identifier associated with the first user.
communication to communicate
with the second user via the For example, Fortnite enables the second user to mute the first user from using voice
network-based portal, based on chat to communicate with the second user via the network-based portal, based on the
the identifier associated with identifier associated with the first user.
the first user;
“You also can ,MUTE players in the party so that you do not hear them through voice
chat.”
https://developer.fortnite.com/help/th/fortnite-c75/battle-royale-c93/how-do-i-manage-
my-party-and-voice-chat-options-from-the-fortnite-lobby-a3477
enabling the first message to be Fortnite also enables the first message to be provided to the second user, using the
provided to the second user, selected mode of communication, depending on the identifier associated with the
using the selected mode of second user, in view of the second user not blocking the first user from using the
communication, depending on selected mode of communication to communicate with the second user, via the
the identifier associated with network-based portal.
the second user, in view of the
second user not blocking the For example, a muted player cannot voice chat with the user, so if the player is not
first user from using the muted, the player will be able to voice chat the user:
selected mode of
communication to communicate “You also can ,MUTE players in the party so that you do not hear them through voice
with the second user, via the chat.”
network-based portal; and
https://developer.fortnite.com/help/th/fortnite-c75/battle-royale-c93/how-do-i-manage-
my-party-and-voice-chat-options-from-the-fortnite-lobby-a3477
7
receiving a second message Fortnite involves receiving a second message from the second user to the first user, to
from the second user to the first respond to the first message, after the second user has received the first message.
user, to respond to the first
message, after the second user “Voice Chat - ON or OFF
has received the first message,
wherein one of the first NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
message or the second message you are playing on may have additional communication features that must be restricted
is voice and the other of the separately. Information on platform-specific controls can be found below or by
first message or the second clicking the “More Settings” button in-game.
message is text,
wherein even when the first In Fortnite, even when the first message is received by the second user via the selected
message is received by the mode of communication, contact information associated with the second user and
second user via the selected provided by the second user to the network-based portal is not provided via the
mode of communication, network-based portal to the first user, and contact information associated with the first
contact information associated user and provided by the first user to the network-based portal is not provided via the
with the second user and network-based portal to the second user, so as to provide an option to the second user
provided by the second user to to keep the contact information associated with the second user confidential from the
8
the network-based portal is not first user, and to provide an option to the first user to keep the contact information
provided via the network-based associated with the first user confidential from the second user.
portal to the first user, and
contact information associated For example, other players cannot see the user’s contact information:
with the first user and provided
by the first user to the network-
based portal is not provided via
the network-based portal to the
second user, so as to provide an
option to the second user to
keep the contact information
associated with the second user
confidential from the first user,
and to provide an option to the
first user to keep the contact https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-c79/how-
information associated with the do-i-change-the-email-on-my-epic-games-account-a3239
first user confidential from the
second user,
wherein the identifier In Fortnite, the identifier associated with the second user is distinct from the contact
associated with the second user information associated with the second user, and the identifier associated with the first
is distinct from the contact user is distinct from the contact information associated with the first user.
information associated with the
second user, and the identifier For example, the user’s identifier (shown in the registration screenshot above as
associated with the first user is “Display Name”) is distinct from the user’s contact information.
distinct from the contact
first user,
wherein the contact information In Fortnite, the contact information associated with the second user includes at least
associated with the second user one of a phone number or an email address of the second user.
includes at least one of a phone
9
number or an email address of

the second user, and
wherein the contact information In Fortnite, the contact information associated with the first user includes at least one
associated with the first user of a phone number or an email address of the first user.
number or an email address of
the first user.
10
2. A computer-implemented Fortnite enables a message to be provided to the second user from the first user, with
method as recited in claim the message based on a mode of communication that is different from the selected
1 comprising enabling a mode, even though the second user has blocked the first user from using the selected
message to be provided to the mode of communication to communicate with the second user.
second user from the first user,
11
with the message based on a For example, as indicated by the following user-submitted suggestions for improving
mode of communication that is Fortnite, a player’s pings will still be received regardless of whether that player is
different from the selected muted:
mode, even though the second
user has blocked the first user “When I mute an annoying player, I want their ping noises muted too.”
from using the selected mode of https://www.reddit.com/r/FortNiteBR/comments/bvaqs2/mute_pings/
with the second user. “i think there should either be a way to mute a player's pings”
https://www.reddit.com/r/FortNiteBR/comments/b2e5f9/add_the_ability_to_mute_
pings_or_make_them_silent/
“Kind of surprised there's no 'mute pings' button yet! The feature is unfinished without
it!”
https://www.reddit.com/r/FortNiteBR/comments/bstt9m/kind_of_surprised_theres_
no_mute_pings_button_yet/
3. A computer-implemented In Fortnite, the mode of communication different from the selected mode includes
method as recited in claim 2, communication with at least an image and communication in an audio manner.
wherein the mode of
communication different from For example, pings include a sound effect and an icon image:
the selected mode includes
communication with at least an “The Fortnite ping system calls out anything in the world with an audible (wait for
image and communication in an it) ping, as well as an identifying icon and distance visible to other people on your
audio manner. team.”
6. A computer-implemented In Fortnite, the plurality of modes of communication supported by the network-based
method as recited in claim 1, portal include at least communication using an electronic device without a computer
wherein the plurality of modes keyboard, and wherein the method supports a user to use an electronic device with a
of communication supported by computer keyboard to communicate with another user using an electronic device
the network-based portal without a computer keyboard.
include at least communication
using an electronic device
12
without a computer keyboard, For example, Fortnite permits a PC user (an electronic device with a computer
and wherein the method keyboard) to communicate with another user using an electronic device without a
supports a user to use an computer keyboard (e.g., a mobile phone).
electronic device with a
computer keyboard to “On Mac and PC, anyone on mobile, PS4 or Xbox One who’s in a cross-platform party
communicate with another user can appear in your match — unless you’re playing in Solo mode, where you'll only
using an electronic device play against other Mac and PC players.”
without a computer keyboard.
7. A computer-implemented Fortnite involves receiving an indication from the second user for the first user via the
method as recited in claim network-based portal to generate an urgent notification for the first user; and
1 comprising: enabling the urgent notification to be generated to notify the first user at least in view
receiving an indication from the of receiving the indication.
second user for the first user via
the network-based portal to “The Fortnite ping system calls out anything in the world with an audible (wait for
generate an urgent notification it) ping, as well as an identifying icon and distance visible to other people on your
for the first user; and team. There's no voice acting involved but it still lets you highlight things like chests,
enabling the urgent notification ammo, weapons and so on, as well as threats, in a way that squad members can easily
to be generated to notify the identify.
first user at least in view of
receiving the indication. In an ideal world it should help squads of randoms coordinate things better without
another thing.
13
squads.”
8. A computer-implemented In Fortnite, the urgent notification is enabled to be generated at least in an audio
method as recited in claim 7, manner.
wherein the urgent notification
is enabled to be generated at “The Fortnite ping system calls out anything in the world with an audible (wait for
least in an audio manner. it) ping, as well as an identifying icon and distance visible to other people on your
identify.
another thing.
squads.”
9. A computer-implemented Fortnite enables an urgent notification to be provided to the second user from the first
method as recited in claim user, even though the second user has blocked the first user from using the selected
1 comprising enabling an urgent mode of communication to communicate with the second user.
notification to be provided to
the second user from the first For example, as indicated by the following user-submitted suggestions for improving
user, even though the second Fortnite, a player’s pings, which can show danger, will still be received regardless of
user has blocked the first user whether that player is muted:
from using the selected mode of
14
communication to communicate “When I mute an annoying player, I want their ping noises muted too.”
with the second user. https://www.reddit.com/r/FortNiteBR/comments/bvaqs2/mute_pings/
“i think there should either be a way to mute a player's pings”

https://www.reddit.com/r/FortNiteBR/comments/b2e5f9/add_the_ability_to_mute_
it!”
15. A computer-implemented Fortnite determines availability of the second user related to receiving messages; and
method as recited in claim 1, sending data to the first user regarding the availability of the second user related to
wherein the method comprises: receiving messages.
determining availability of the
second user related to receiving For example, the game shows whether the user’s friends are online or offline:
messages; and
sending data to the first user
regarding the availability of the
second user related to receiving
messages.
https://www.newsweek.com/fortnite-mobile-how-add-friends-send-invite-codes-beta-
850171#slideshow/850160
16. A computer-implemented Fortnite enables the second user to select a predetermined message to send to the first
method as recited in claim user, the predetermined message being selected from a set of predetermined messages
1 comprising enabling the provided to the second user.
second user to select a
predetermined message to send For example, the user can use the ping system to send different types of pings, such as
to the first user, the a danger ping:
15
predetermined message being

selected from a set of “The Fortnite ping system calls out anything in the world with an audible (wait for
predetermined messages it) ping, as well as an identifying icon and distance visible to other people on your
provided to the second user. team.
squads.”
17. A computer-implemented In Fortnite, the plurality of communication modes include at least text communication
method as recited in claim using a mobile phone, and voice communication using a mobile phone.
1 wherein the plurality of
communication modes include
at least text communication “Here’s how to chat cross-platform and talk cross-platform with players on PC, PS4,
using a mobile phone, and voice Xbox One, and mobile.”
communication using a mobile
phone. https://twinfinite.net/2018/09/fortnite-cross-platform-chat/
16
EXHIBIT F
Claim 46
A non-transitory computer As shown below, Fortnite utilizes a non-transitory computer readable medium
readable medium including at including at least executable computer program code stored therein that facilitates
least executable computer electronic communication of a plurality of users using at least a network-based portal at
program code stored therein least based on Internet protocol, with a plurality of modes of communication available
that facilitates electronic for the plurality of users to communicate, with each of the plurality of users having an
communication of a plurality of identifier for use with the plurality of modes of communication, and without requiring
users using at least a network- the plurality of users to disclose their contact information to each other.
based portal at least based on
Internet protocol, with a
communication available for
the plurality of users to
communicate, with each of the
plurality of users having an
identifier for use with the
communication, and without
requiring the plurality of users
to disclose their contact
information to each other, the
computer readable medium
comprising:
computer program code for Fortnite utilizes computer program code for providing a plurality of modes of
providing a plurality of modes communication to a first user to allow the first user to use one of the plurality of modes
of communication to a first user of communication as a selected mode of communication for a first message to be sent
to allow the first user to use one from the first user to a second user, based on an identifier associated with the first user
of the plurality of modes of previously set by the first user via the network-based portal.
communication as a selected
mode of communication for a
1
first message to be sent from For example, PC users will be directed to a registration screen as shown below. So,
the first user to a second user, players must be registered with their ID (DISPLAY NAME) to use the game (and,
based on an identifier therefore, to send messages in any mode available within the game):
associated with the first user
previously set by the first user
2
“Voice Chat - ON or OFF
NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
wherein the plurality of modes The plurality of modes of communication supported by the network-based portal
of communication supported by utilized by Fortnite include at least text communication using a personal computer,
the network-based portal voice communication using a personal computer, text communication using a mobile
include at least text phone, voice communication using a mobile phone.
communication using a
personal computer, voice For example, the parental controls shown below, which show both voice and text chat
communication using a options, are available on all platforms, including PC and mobile:
personal computer, text
communication using a mobile “Voice Chat - ON or OFF
3
phone, voice communication NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
using a mobile phone, and you are playing on may have additional communication features that must be restricted
“Where can I download Battle Royale?

You can download Battle Royale in the following places:
PC/Mac - Fortnite.com (Apple has terminated our ability to develop Fortnite for Mac.
As a result, Fortnite on Mac remains on version 13.40 for Battle Royale/Creative.)
PlayStation - PlayStation Store
Xbox - Xbox Marketplace
Nintendo Switch - Nintendo eShop
Android - Currently available through the Epic Games App on the Samsung Galaxy
Store or epicgames.com. Google has blocked your access to Fortnite on Google Play.
iOS - Apple has blocked your access to Fortnite on iOS devices.”
4
https://www.epicgames.com/fortnite/en-US/faq
“On Mac and PC, anyone on mobile, PS4 or Xbox One who’s in a cross-platform party
can appear in your match — unless you’re playing in Solo mode, where you'll only
play against other Mac and PC players.”
communication with at least an Fortnite utilizes communication with at least an image.

image, and
“The Fortnite ping system calls out anything in the world with an audible (wait for
it) ping, as well as an identifying icon and distance visible to other people on your
identify.
another thing.
squads.”
wherein messages are eligible In Fortnite, messages are eligible to be received electronically by the second user via
to be received electronically by the network-based portal, based on any of the plurality of modes of communication, all
the second user via the depending on an identifier associated with the second user previously set by the second
5
network-based portal, based on user via the network-based portal, which allows the second user to efficiently maintain
any of the plurality of modes of the second user’s communication using the plurality of modes of communication.
communication, all depending
on an identifier associated with For example, players must be registered with their ID to use the game (and, therefore,
the second user previously set to receive messages in any mode available within the game):
by the second user via the
network-based portal, which
allows the second user to
efficiently maintain the second
user’s communication using the
communication;
6
computer program code for Fortnite utilizes computer program code for permitting the second user to block the
permitting the second user to first user from using at least the selected mode of communication to communicate with
block the first user from using the second user via the network-based portal, based on the identifier associated with
at least the selected mode of the first user.
with the second user via the For example, Fortnite permits the second user to block the first user from using voice
network-based portal, based on chat as the selected mode of communication to communicate with the second user.
the identifier associated with
the first user; Fortnite also utilizes computer program code for enabling the first message to be
computer program code for electronically provided to the second user, using voice chat as the selected mode of
enabling the first message to be communication, depending on the identifier associated with the second user, in view of
electronically provided to the the second user not muting the first user from using voice chat to communicate with
second user, using the selected the second user, via the network-based portal.
mode of communication,
depending on the identifier For example, a muted player cannot voice chat the user, so if the player is not muted,
associated with the second user, the player will be able to voice chat with the user:
in view of the second user not
blocking the first user from “You also can ,MUTE players in the party so that you do not hear them through voice
using the selected mode of chat.”
with the second user, via the https://developer.fortnite.com/help/th/fortnite-c75/battle-royale-c93/how-do-i-manage-
network-based portal; my-party-and-voice-chat-options-from-the-fortnite-lobby-a3477
computer program code for Fortnite utilizes computer code for determining availability of the second user related
determining availability of the to receiving messages.
second user related to receiving
messages; and For example, the game shows whether the user’s friends are online or offline:
7
850171#slideshow/850160
computer program code for Fortnite utilizes computer program code for receiving, from the second user, contact
receiving, from the second user, information associated with the second user to allow the second user to participate and
contact information associated at least receive messages via the network-based portal.
with the second user to allow
the second user to participate For example, during registration, the portal receives the user’s contact information to
and at least receive messages allow the user to play Fortnite, and therefore to receive messages within Fortnite:
8
wherein even when the first In Fortnite, even when the first message is received by the second user via the selected
message is received by the mode of communication, the contact information associated with the second user is not
second user via the selected provided via the network-based portal to the first user, and contact information
mode of communication, the associated with the first user is not provided via the network-based portal to the second
contact information associated user, so as to provide an option to the second user to keep the contact information
9
with the second user is not associated with the second user confidential from the first user, and to provide an
provided via the network-based option to the first user to keep the contact information associated with the first user
portal to the first user, and confidential from the second user.
contact information associated
with the first user is not For example, other players cannot see the user’s contact information:
provided via the network-based
portal to the second user, so as
to provide an option to the
second user to keep the contact
second user confidential from
the first user, and to provide an
option to the first user to keep
the contact information
associated with the first user https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-
confidential from the second c79/how-do-i-change-the-email-on-my-epic-games-account-a3239
user, and
wherein the identifier In Fortnite, the identifier associated with the second user is distinct from the contact
associated with the second user information associated with the second user, and the identifier associated with the first
is distinct from the contact user is distinct from the contact information associated with the first user.
second user, and the identifier For example, the user’s identifier (shown in the registration screenshot above as
associated with the first user is “Display Name”) is distinct from the user’s contact information.
distinct from the contact
first user.
Claim 58
A non-transitory computer Fortnite utilizes computer program code for receiving a second message from the
readable medium as recited in second user to the first user, with one of the first message or the second message being
claim 46, voice and the other one of the first message or the second message being text.
wherein said computer
readable medium comprises
10
computer program code for For example, players can use both voice and text chat in the game, as seen in the
receiving a second message settings page shown below:
from the second user to the first
user, with one of the first “Voice Chat - ON or OFF
message or the second message
being voice and the other one of NOTE: This setting only enables/disables Fortnite’s in-game voice chat. The platform
the first message or the second you are playing on may have additional communication features that must be restricted
message being text, and separately. Information on platform-specific controls can be found below or by
11
wherein the contact information In Fortnite, the contact information associated with the second user includes at least
associated with the second user one of a phone number or an email address of the second user.
number or an email address of For example, during registration, the portal receives the user’s email address:
the second user.
12
Claim 59
A non-transitory computer Fortnite utilizes computer program code for enabling the second user to select a
readable medium as recited in predetermined message to send to the first user via at least one of the plurality of
claim 58, wherein said modes of communication, the predetermined message being selected from a set of
computer readable medium predetermined messages provided to the second user.
comprises computer program
code for enabling the second For example, the user can use the ping system to send different types of pings, such as
user to select a predetermined a danger ping:
message to send to the first user
via at least one of the plurality “The Fortnite ping system calls out anything in the world with an audible (wait for
of modes of communication, it) ping, as well as an identifying icon and distance visible to other people on your
the predetermined message team.
being selected from a set of …
predetermined messages
provided to the second user. The Fortnite ping system lives on the left D-pad button. When you press that it'll ping
squads.”
Claim 60
A non-transitory computer Fortnite utilizes computer program code for enabling a message to be provided to the
readable medium as recited in second user from the first user, with the message based on a mode of communication
claim 59, the computer readable that is different from the selected mode, even though the second user has blocked the
medium comprises computer first user from using the selected mode of communication to communicate the first
program code for enabling a message with the second user.
message to be provided to the
second user from the first user,
13
with the message based on a For example, as indicated by the following user-submitted suggestions for improving
mode of communication that is Fortnite, a player’s pings will still be received regardless of whether that player is
different from the selected muted:
mode, even though the second
user has blocked the first user “When I mute an annoying player, I want their ping noises muted too.”
from using the selected mode of https://www.reddit.com/r/FortNiteBR/comments/bvaqs2/mute_pings/
the first message with the “i think there should either be a way to mute a player's pings”
second user. https://www.reddit.com/r/FortNiteBR/comments/b2e5f9/add_the_ability_to_mute_
it!”
Claim 61
A non-transitory computer In Fortnite, the mode of communication different from the selected mode includes
readable medium as recited in communication with at least an image and communication in an audio manner.
claim 60, wherein the mode of
communication different from For example, pings include a sound effect and an icon image:
the selected mode includes
communication with at least an “The Fortnite ping system calls out anything in the world with an audible (wait for
image and communication in an it) ping, as well as an identifying icon and distance visible to other people on your
audio manner. team.”
Claim 62
A non-transitory computer In Fortnite, one of the electronic devices associated with the first user or the electronic
readable medium as recited in device associated with the second user is a personal computer, and
claim 61, the other one of the electronic device associated with the first user or the electronic
wherein one of the electronic device associated with the second user is a mobile phone.
device associated with the first
14
user or the electronic device Users on PC can play the game and communicate with users on mobile:
is a personal computer, and “On Mac and PC, anyone on mobile, PS4 or Xbox One who’s in a cross-platform party
wherein the other one of the can appear in your match — unless you’re playing in Solo mode, where you'll only
electronic device associated play against other Mac and PC players.”
with the first user or the
electronic device associated https://www.polygon.com/2018/3/23/17146848/cross-platform-crossplay-ps4-xbox-pc-
with the second user is a mobile switch-ios-mobile-enable-friends-compatible-matchmaking
phone.
15
EXHIBIT G
1. A computer-implemented As shown below, Fortnite involves a computer-implemented method for managing

method for managing electronic electronic communications using at least a network-based portal at least based on
communications using at least a Internet protocol.
network-based portal at least
based on Internet protocol, the
method comprising:
providing a plurality of Fortnite provides a plurality of communication options to a first user to be selected as a
communication options to a selected option of communication for a message from the first user to a second user via
first user to be selected as a an electronic device associated with the second user, with the first user being identified
selected option of at least depending on a prior registration process by the first user regarding the use of
communication for a message the network-based portal, and with the plurality of communication options provided to
from the first user to a second the first user to send messages to the electronic device associated with the second user.
user via an electronic device
associated with the second user, For example, Fortnite provides a plurality of communication options for a first user to
with the first user being select:
identified at least depending on
a prior registration process by “Both voice chat and on-screen text chat are options.”
the first user regarding the use
of the network-based portal, https://www.commonsensemedia.org/blog/parents-ultimate-guide-to-fortnite
and with the plurality of
communication options PC users will be directed to a registration screen as shown below. So, players must be
provided to the first user to send registered with their ID (DISPLAY NAME) to use the game (and, therefore, to send
messages to the electronic messages in any mode available within the game):
device associated with the
second user,
1
wherein the plurality of Fortnite’s plurality of communication options include text messaging and voice
communication options include communication.
text messaging and voice
communication, and “Both voice chat and on-screen text chat are options.”
2
https://www.commonsensemedia.org/blog/parents-ultimate-guide-to-fortnite
wherein all of the In Fortnite, all of the communication options use one identifier associated with the
communication options use one second user for the second user to receive messages, at least in view of the network-
identifier associated with the based portal being based on the Internet protocol.
second user for the second user
to receive messages, at least in For example, players must be registered with their ID to use the game (and, therefore,
view of the network-based to message within the game):
portal being based on the
Internet protocol;
3
receiving an indication Fortnite receives an indication regarding one of the plurality of communication
regarding one of the plurality of options, via the network-based portal, from an electronic device associated with the
communication options, via the first user, the indication indicating the selected option of communication for the
network-based portal, from an message from the plurality of communication options provided.
electronic device associated
4
with the first user, the For example, Fortnite’s portal receives an indication of a selected communication
indication indicating the option from a first user’s device:
selected option of
communication for the message
from the plurality of
communication options
provided;
http://www.tomsguide.com/faq/id-3738546/fortnite-battle-royale-basic-controls-
player.html
permitting the second user to Fortnite permits the second user to block the first user from reaching the second user
block the first user from via the network-based portal.
reaching the second user via the
network-based portal; and https://www.youtube.com/watch?v=V51C14CclkI
enabling, via the network-based On information and belief, Fortnite enables, via the network-based portal, the message
portal, the message to be to be received by the second user through the electronic device associated with the
received by the second user second user, using the selected option of communication, based on the one identifier
through the electronic device associated with the second user, in view of the second user not blocking the first user
associated with the second user, from reaching the second user, wherein a piece of information regarding the second
using the selected option of user blocking the first user from reaching the second user is stored in a storage medium
communication, based on the if the second user has blocked the first user from reaching the second user, with the
one identifier associated with
5
the second user, in view of the piece of information being based on at least an input previously submitted by the
second user not blocking the second user.
first user from reaching the
second user, wherein a piece of For example, as discussed above, a second user can receive a message through his or
information regarding the her device using the first user’s selected option of communication.
second user blocking the first
user from reaching the second The second user receives messages based on the second user’s ID as long as the second
user is stored in a storage user hasn’t blocked the first user.
medium if the second user has
blocked the first user from https://www.youtube.com/watch?v=V51C14CclkI
reaching the second user, with
the piece of information being On information and belief, information regarding blocked users is stored in a storage
based on at least an input medium based on input submitted by the second user.
previously submitted by the
second user,
wherein the method comprises Based on information and belief, Fortnite determines availability of the second user.
determining availability of the
second user, For example, the game shows whether a user’s friends are online or offline:
850171#slideshow/850160
wherein the method requires Fortnite requires contact information associated with the second user to allow the
contact information associated second user to receive messages via the network-based portal.
6
with the second user to allow

the second user to receive For example, a second user must provide contact information to the portal in order to
messages via the network-based receive messages:
portal,
7
wherein even when the message On information and belief, even when the message is received by the second user
is received by the second user through the electronic device associated with the second user based on the one
through the electronic device identifier associated with the second user, the contact information associated with the
associated with the second user second user is not provided via the network-based portal to the first user through the
based on the one identifier electronic device associated with the first user.
associated with the second user,
is not provided via the network-
based portal to the first user
through the electronic device
associated with the first user,
and
https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-
c79/how-do-i-change-the-email-on-my-epic-games-account-a3239
wherein the one identifier On information and belief, the one identifier associated with the second user is distinct
associated with the second user from the contact information associated with the second user.
is distinct from the contact
information associated with the For example, the second user’s ID is not the same as the user’s contact info.
second user.
850171#slideshow/850160
8
11. A computing apparatus for As shown below, Fortnite utilizes a computing apparatus for managing electronic
managing electronic communications using at least a network-based portal at least based on Internet
communications using at least a protocol.
network-based portal at least
based on Internet protocol, the
computing apparatus
comprising:
at least one computing device; As evident from the evidence provided below, Fortnite utilizes at least one computing
and device and one or more storage devices coupled to the at least one computing device,
one or more storage devices with the one or more storage devices storing instructions that cause the computing
coupled to the at least one apparatus to practice the remaining elements of this claim.
computing device, with the one
or more storage devices storing
instructions that, when
executed, cause the computing
apparatus to:
providing a plurality of Fortnite provides a plurality of communication options to a first user to be selected as a
communication options to a selected option of communication for a message from the first user to a second user via
first user to be selected as a an electronic device associated with the second user, with the first user being identified
selected option of by the computing apparatus at least depending on a prior registration process by the
9
communication for a message first user regarding the use of the network-based portal, and with the plurality of
from the first user to a second communication options provided to the first user to send messages to the electronic
user via an electronic device device associated with the second user.
with the first user being For example, Fortnite provides a plurality of communication options for a first user to
identified by the computing select:
apparatus at least depending on
a prior registration process by “Both voice chat and on-screen text chat are options.”
the first user regarding the use
of the network-based portal, https://www.commonsensemedia.org/blog/parents-ultimate-guide-to-fortnite
and with the plurality of
communication options Fortnite provides a portal through which users register (e.g., the below example for PC
provided to the first user to send users):
messages to the electronic
device associated with the
second user,
10
wherein the plurality of Fortnite’s plurality of communication options include text messaging and voice
communication options include communication.
text messaging and voice
communication, and “Both voice chat and on-screen text chat are options.”
11
https://www.commonsensemedia.org/blog/parents-ultimate-guide-to-fortnite
wherein all of the Fortnite’s communication options all use one identifier associated with the second user
communication options use one for the second user to receive messages, at least in view of the network-based portal
identifier associated with the being based on the Internet protocol.
second user for the second user
to receive messages, at least in For example, players must be registered with their ID to use the game (and, therefore,
view of the network-based to message within the game):
portal being based on the
Internet protocol;
12
receiving an indication Fortnite receives an indication regarding one of the plurality of communication
regarding one of the plurality of options, via the network-based portal, from an electronic device associated with the
communication options, via the first user, the indication indicating the selected option of communication for the
network-based portal, from an message from the plurality of communication options provided.
electronic device associated
13
with the first user, the For example, Fortnite’s portal receives an indication of a selected communication
indication indicating the option from a first user’s device:
selected option of
communication for the message
from the plurality of
communication options
provided;
http://www.tomsguide.com/faq/id-3738546/fortnite-battle-royale-basic-controls-
player.html
permitting the second user to Fortnite permits the second user to block the first user from reaching the second user
block the first user from via the network-based portal.
reaching the second user via the
network-based portal; and https://www.youtube.com/watch?v=V51C14CclkI
enabling, via the network-based Fortnite enables, via the network-based portal, the message to be received by the
portal, the message to be second user through the electronic device associated with the second user, using the
received by the second user selected option of communication, based on the one identifier associated with the
through the electronic device second user.
using the selected option of For example, as discussed above, a second user can receive a message through his or
communication, based on the her device using the first user’s selected option of communication.
one identifier associated with
the second user, in view of the The second user receives messages based on the second user’s ID as long as the second
second user not blocking the user hasn’t blocked the first user from reaching the second user.
14
first user from reaching the https://www.youtube.com/watch?v=V51C14CclkI

second user, wherein a piece of
information regarding the On information and belief, information regarding blocked users is stored in a storage
second user blocking the first medium based on input submitted by the second user.
user from reaching the second
user is stored in a storage
medium if the second user has
blocked the first user from
reaching the second user, with
the piece of information being
based on at least an input
previously submitted by the
second user,
wherein the instructions, when Fortnite’s instructions, when executed, cause the computing apparatus to determine
executed, cause the computing availability of the second user, and to require contact information associated with the
apparatus to determine second user to allow the second user to receive messages via the network-based portal.
availability of the second user,
and to require contact For example, the game shows whether a user’s friends are online or offline:
second user to allow the second
user to receive messages via the
network-based portal,
850171#slideshow/850160
A second user must provide contact information to the portal in order to receive
messages:
15
wherein even when the message In Fortnite, even when the message is received by the second user through the
is received by the second user electronic device associated with the second user based on the one identifier associated
through the electronic device with the second user, the contact information associated with the second user is not
associated with the second user provided via the network-based portal to the first user through the electronic device
based on the one identifier associated with the first user.
16

is not provided via the network-
based portal to the first user
through the electronic device
associated with the first user,
and
wherein the one identifier In Fortnite, the one identifier associated with the second user is distinct from the
associated with the second user contact information associated with the second user.
is distinct from the contact
information associated with the For example, the second user’s ID is not the same as the user’s contact info.
second user.
850171#slideshow/850160
17
18
EXHIBIT H
Claim 53
A server based on a As shown below, in connection with its Fortnite video game, Epic Games provides a server based on a
communication communication protocol for managing the communication of a user, in view of a person, using a first device,
protocol for trying to electronically convey a message from the first device to a second device of the user, the second
managing the device being a handheld communication device, the server comprising:
communication of a at least one computing device; and
user, in view of a at least one storage device,
person, using a first wherein the at least one computing device is configured to
device, trying to receive the message from the person using the first device.
electronically
convey a message
from the first
device to a second
device of the user,
the second device
being a handheld
communication
device, the server
comprising:
at least one
computing device;
and
at least one storage
device,
wherein the at least
one computing
device is
configured to
receive the message
from the person
using the first
device;
identify the person Epic Games identifies the person attempting to electronically convey the message.
attempting to
1
electronically For example, users are identified with a display name:

convey the
message; and “Your Epic display name is the name that appears for your account on the Epic Games Launcher, our official
forums, AnswerHub, and when you play any of our games on PC/Mac, Switch, or mobile.”
https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-c79/how-to-change-your-epic-
games-display-name-a3260
set a process for the Epic Games sets a process for the message using one or more rules based on at least a status associated with
message using one the user and an access priority associated with the person, the status depending at least in part on the current
or more rules based activity or location of the user, or the current time, and the access priority depending on the person.
on at least a status
associated with the For example, processes for Fortnite’s different modes of communication can be based on the sender’s access
user and an access priority.
priority associated
with the person, the The sender needs to be on the user’s friends list before the user could get any messages from the sender.
status depending at
least in part on the The user can lower another player’s access priority by muting the another player’s voice chat, which prevents
current activity or the another player from voice chatting with the user.
location of the user,
or the current time, “You also can ,MUTE players in the party so that you do not hear them through voice chat.”
and the access https://developer.fortnite.com/help/th/fortnite-c75/battle-royale-c93/how-do-i-manage-my-party-and-voice-
priority depending chat-options-from-the-fortnite-lobby-a3477
on the person,
During gameplay, enemy players automatically have lower access priority, since only teammates can send
pings to the user:
“The Fortnite ping system calls out anything in the world with an audible (wait for it) ping, as well as an
identifying icon and distance visible to other people on your team.”
A process for the message can be based on the user’s current activity. For example, if the user is currently
playing the game, the user can receive pings from the user’s teammates:
“The Fortnite ping system calls out anything in the world with an audible (wait for it) ping, as well as an
identifying icon and distance visible to other people on your team.”
https://www.gamesradar.com/fortnite-ping-system-how-to/.
2
Pings are available only during gameplay (the “world” referred to above is the world of the game), so the
process for ping messages requires the user’s current activity to be “playing the game.”
wherein the server Epic Games’ server is configured to have access to contact information of the person.
is configured to
have access to
contact information
of the person,
https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-c79/how-do-i-change-the-
email-on-my-epic-games-account-a3239
wherein even if the Epic Games’ server does not provide the contact information of the user to the first device to inhibit the person
process includes from sending messages to the user without via the server.
transmitting the
message to the For example, other players cannot see the user’s contact information:
second device via
contact information
of the user, the
server does not
provide the contact
information of the
user to the first
device to inhibit the
person from
sending messages
to the user without https://www.epicgames.com/help/en-US/epic-accounts-c74/general-support-c79/how-do-i-change-the-
via the server, email-on-my-epic-games-account-a3239
wherein the server Epic Games’ server is configured to be aware of the availability of the user.
is configured to be
aware of the For example, the server is aware of whether the user is online or offline, and whether the user is in the Epic
availability of the launcher (indicated by the Epic icon), or in the game itself (which would be indicated by a Fortnite icon).
user, and
3
“What Epic Account information does Epic share with other services?
Other services can request the following information associated with your account:
• Your display name

To learn how to review and update your display name, see How do I change my display name?
• Your linked accounts
You can review the accounts you have linked on the account management portal.
• Your online status, which reflects whether you’re offline or online, and also gives the game you're
currently playing with your Epic Account
• List of your Epic friends who have also authorized this game to see them
You will be asked for explicit consent about each of these pieces of information before a product or service
can receive access to them.”
https://www.epicgames.com/help/en-US/epic-accounts-c74/epic-account-services-c144/what-epic-account-
information-does-epic-share-with-other-services-a4180
4
https://www.reddit.com/r/FortniteCompetitive/comments/cx9uru/makes_me_sad_to_see_a_99_person_friends
_list_with/
wherein the server Epic Games’ server is configured to send information to the person regarding the availability of the user.
is configured to
send information to For example, the server displays this information to other players who have the user in their contact list, as
the person shown above.
regarding the
availability of the
user.
5
JS 44 (Rev. 10/20) Case 6:21-cv-00663-ADA
CIVILDocument
COVER1-9 Filed 06/25/21 Page 1 of 2
SHEET
The JS 44 civil cover sheet and the information contained herein neither replace nor supplement the filing and service of pleadings or other papers as required by law, except as
provided by local rules of court. This form, approved by the Judicial Conference of the United States in September 1974, is required for the use of the Clerk of Court for the
purpose of initiating the civil docket sheet. (SEE INSTRUCTIONS ON NEXT PAGE OF THIS FORM.)
I. (a) PLAINTIFFS DEFENDANTS
IngenioShare, LLC (Civil Case No. 6:21-cv-00663) Epic Games, Inc.
(b) County of Residence of First Listed Plaintiff Santa Clara County, CA County of Residence of First Listed Defendant
(EXCEPT IN U.S. PLAINTIFF CASES) (IN U.S. PLAINTIFF CASES ONLY)
NOTE: IN LAND CONDEMNATION CASES, USE THE LOCATION OF
THE TRACT OF LAND INVOLVED.
(c) Attorneys (Firm Name, Address, and Telephone Number) Attorneys (If Known)
Cortney S. Alexander, Kent & Risley LLC
5755 N Point Pkwy, Ste 57, Alpharetta, GA 30022
Tel: 404.855.3867
II. BASIS OF JURISDICTION (Place an “X” in One Box Only) III. CITIZENSHIP OF PRINCIPAL PARTIES (Place an “X” in One Box for Plaintiff
(For Diversity Cases Only) and One Box for Defendant)
1 U.S. Government ✖ 3 Federal Question PTF DEF PTF DEF
Plaintiff (U.S. Government Not a Party) Citizen of This State 1 1 Incorporated or Principal Place 4 4
of Business In This State
2 U.S. Government 4 Diversity Citizen of Another State 2 2 Incorporated and Principal Place 5 5
Defendant (Indicate Citizenship of Parties in Item III) of Business In Another State
Citizen or Subject of a 3 3 Foreign Nation 6 6

Foreign Country
IV. NATURE OF SUIT (Place an “X” in One Box Only) Click here for: Nature of Suit Code Descriptions.
CONTRACT TORTS FORFEITURE/PENALTY BANKRUPTCY OTHER STATUTES
110 Insurance PERSONAL INJURY PERSONAL INJURY 625 Drug Related Seizure 422 Appeal 28 USC 158 375 False Claims Act
120 Marine 310 Airplane 365 Personal Injury - of Property 21 USC 881 423 Withdrawal 376 Qui Tam (31 USC
130 Miller Act 315 Airplane Product Product Liability 690 Other 28 USC 157 3729(a))
140 Negotiable Instrument Liability 367 Health Care/ 400 State Reapportionment
150 Recovery of Overpayment 320 Assault, Libel & Pharmaceutical PROPERTY RIGHTS 410 Antitrust
& Enforcement of Judgment Slander Personal Injury 820 Copyrights 430 Banks and Banking
151 Medicare Act 330 Federal Employers’ Product Liability ✖ 830 Patent 450 Commerce
152 Recovery of Defaulted Liability 368 Asbestos Personal 835 Patent - Abbreviated 460 Deportation
Student Loans 340 Marine Injury Product New Drug Application 470 Racketeer Influenced and
(Excludes Veterans) 345 Marine Product Liability 840 Trademark Corrupt Organizations
153 Recovery of Overpayment Liability PERSONAL PROPERTY LABOR 880 Defend Trade Secrets 480 Consumer Credit
of Veteran’s Benefits 350 Motor Vehicle 370 Other Fraud 710 Fair Labor Standards Act of 2016 (15 USC 1681 or 1692)
160 Stockholders’ Suits 355 Motor Vehicle 371 Truth in Lending Act 485 Telephone Consumer
190 Other Contract Product Liability 380 Other Personal 720 Labor/Management SOCIAL SECURITY Protection Act
195 Contract Product Liability 360 Other Personal Property Damage Relations 861 HIA (1395ff) 490 Cable/Sat TV
196 Franchise Injury 385 Property Damage 740 Railway Labor Act 862 Black Lung (923) 850 Securities/Commodities/
362 Personal Injury - Product Liability 751 Family and Medical 863 DIWC/DIWW (405(g)) Exchange
Medical Malpractice Leave Act 864 SSID Title XVI 890 Other Statutory Actions
REAL PROPERTY CIVIL RIGHTS PRISONER PETITIONS 790 Other Labor Litigation 865 RSI (405(g)) 891 Agricultural Acts
210 Land Condemnation 440 Other Civil Rights Habeas Corpus: 791 Employee Retirement 893 Environmental Matters
220 Foreclosure 441 Voting 463 Alien Detainee Income Security Act FEDERAL TAX SUITS 895 Freedom of Information
230 Rent Lease & Ejectment 442 Employment 510 Motions to Vacate 870 Taxes (U.S. Plaintiff Act
240 Torts to Land 443 Housing/ Sentence or Defendant) 896 Arbitration
245 Tort Product Liability Accommodations 530 General 871 IRS—Third Party 899 Administrative Procedure
290 All Other Real Property 445 Amer. w/Disabilities - 535 Death Penalty IMMIGRATION 26 USC 7609 Act/Review or Appeal of
Employment Other: 462 Naturalization Application Agency Decision
446 Amer. w/Disabilities - 540 Mandamus & Other 465 Other Immigration 950 Constitutionality of
Other 550 Civil Rights Actions State Statutes
448 Education 555 Prison Condition
560 Civil Detainee -
Conditions of
Confinement
V. ORIGIN (Place an “X” in One Box Only)
✖ 1 Original 2 Removed from 3 Remanded from 4 Reinstated or 5 Transferred from 6 Multidistrict 8 Multidistrict
Proceeding State Court Appellate Court Reopened Another District Litigation - Litigation -
(specify) Transfer Direct File
Cite the U.S. Civil Statute under which you are filing (Do not cite jurisdictional statutes unless diversity):
35 U.S.C. Sections 271, 281, 284, and 285
VI. CAUSE OF ACTION Brief description of cause:
Patent infringement
VII. REQUESTED IN CHECK IF THIS IS A CLASS ACTION DEMAND $ CHECK YES only if demanded in complaint:
COMPLAINT: UNDER RULE 23, F.R.Cv.P. JURY DEMAND: ✖ Yes No
VIII. RELATED CASE(S)
(See instructions):
IF ANY JUDGE DOCKET NUMBER
DATE SIGNATURE OF ATTORNEY OF RECORD
Digitally signed by Cortney S. Alexander
06/25/2021 Cortney S. Alexander Date: 2021.06.25 10:55:04 -05'00'
FOR OFFICE USE ONLY
RECEIPT # AMOUNT APPLYING IFP JUDGE MAG. JUDGE

JS 44 Reverse (Rev. 10/20) Case 6:21-cv-00663-ADA Document 1-9 Filed 06/25/21 Page 2 of 2
INSTRUCTIONS FOR ATTORNEYS COMPLETING CIVIL COVER SHEET FORM JS 44
Authority For Civil Cover Sheet
The JS 44 civil cover sheet and the information contained herein neither replaces nor supplements the filings and service of pleading or other papers as
required by law, except as provided by local rules of court. This form, approved by the Judicial Conference of the United States in September 1974, is
required for the use of the Clerk of Court for the purpose of initiating the civil docket sheet. Consequently, a civil cover sheet is submitted to the Clerk of
Court for each civil complaint filed. The attorney filing a case should complete the form as follows:
I.(a) Plaintiffs-Defendants. Enter names (last, first, middle initial) of plaintiff and defendant. If the plaintiff or defendant is a government agency, use
only the full name or standard abbreviations. If the plaintiff or defendant is an official within a government agency, identify first the agency and then
the official, giving both name and title.
(b) County of Residence. For each civil case filed, except U.S. plaintiff cases, enter the name of the county where the first listed plaintiff resides at the
time of filing. In U.S. plaintiff cases, enter the name of the county in which the first listed defendant resides at the time of filing. (NOTE: In land
condemnation cases, the county of residence of the "defendant" is the location of the tract of land involved.)
(c) Attorneys. Enter the firm name, address, telephone number, and attorney of record. If there are several attorneys, list them on an attachment, noting
in this section "(see attachment)".
II. Jurisdiction. The basis of jurisdiction is set forth under Rule 8(a), F.R.Cv.P., which requires that jurisdictions be shown in pleadings. Place an "X"
in one of the boxes. If there is more than one basis of jurisdiction, precedence is given in the order shown below.
United States plaintiff. (1) Jurisdiction based on 28 U.S.C. 1345 and 1348. Suits by agencies and officers of the United States are included here.
United States defendant. (2) When the plaintiff is suing the United States, its officers or agencies, place an "X" in this box.
Federal question. (3) This refers to suits under 28 U.S.C. 1331, where jurisdiction arises under the Constitution of the United States, an amendment
to the Constitution, an act of Congress or a treaty of the United States. In cases where the U.S. is a party, the U.S. plaintiff or defendant code takes
precedence, and box 1 or 2 should be marked.
Diversity of citizenship. (4) This refers to suits under 28 U.S.C. 1332, where parties are citizens of different states. When Box 4 is checked, the
citizenship of the different parties must be checked. (See Section III below; NOTE: federal question actions take precedence over diversity
cases.)
III. Residence (citizenship) of Principal Parties. This section of the JS 44 is to be completed if diversity of citizenship was indicated above. Mark this
section for each principal party.
IV. Nature of Suit. Place an "X" in the appropriate box. If there are multiple nature of suit codes associated with the case, pick the nature of suit code
that is most applicable. Click here for: Nature of Suit Code Descriptions.
V. Origin. Place an "X" in one of the seven boxes.

Original Proceedings. (1) Cases which originate in the United States district courts.
Removed from State Court. (2) Proceedings initiated in state courts may be removed to the district courts under Title 28 U.S.C., Section 1441.
Remanded from Appellate Court. (3) Check this box for cases remanded to the district court for further action. Use the date of remand as the filing
date.
Reinstated or Reopened. (4) Check this box for cases reinstated or reopened in the district court. Use the reopening date as the filing date.
Transferred from Another District. (5) For cases transferred under Title 28 U.S.C. Section 1404(a). Do not use this for within district transfers or
multidistrict litigation transfers.
Multidistrict Litigation – Transfer. (6) Check this box when a multidistrict case is transferred into the district under authority of Title 28 U.S.C.
Section 1407.
Multidistrict Litigation – Direct File. (8) Check this box when a multidistrict case is filed in the same district as the Master MDL docket.
PLEASE NOTE THAT THERE IS NOT AN ORIGIN CODE 7. Origin Code 7 was used for historical records and is no longer relevant due to
changes in statue.
VI. Cause of Action. Report the civil statute directly related to the cause of action and give a brief description of the cause. Do not cite jurisdictional
statutes unless diversity. Example: U.S. Civil Statute: 47 USC 553 Brief Description: Unauthorized reception of cable service.
VII. Requested in Complaint. Class Action. Place an "X" in this box if you are filing a class action under Rule 23, F.R.Cv.P.
Demand. In this space enter the actual dollar amount being demanded or indicate other demand, such as a preliminary injunction.
Jury Demand. Check the appropriate box to indicate whether or not a jury is being demanded.
VIII. Related Cases. This section of the JS 44 is used to reference related pending cases, if any. If there are related pending cases, insert the docket
numbers and the corresponding judge names for such cases.
Date and Attorney Signature. Date and sign the civil cover sheet.

In The United States District Court Western District of Texas Waco Division

Uploaded by

Copyright:

Available Formats

In The United States District Court Western District of Texas Waco Division

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

In The United States District Court Western District of Texas Waco Division

Uploaded by

Copyright:

Available Formats

Case 6:21-cv-00663-ADA Document 1 Filed 06/25/21 Page 1 of 29

IN THE UNITED STATES DISTRICT COURT

EPIC GAMES, INC., JURY TRIAL DEMANDED

COMPLAINT FOR PATENT INFRINGEMENT

Games, Inc. as follows:

the United States, Title 35, United States Code.

2. Plaintiff IngenioShare, LLC is a limited liability company organized

and existing under laws of California.

3. On information and belief, Defendant Epic Games, Inc. (“Defendant”)

is a corporation organized under the laws of Maryland, having a place of business

900, Dallas, TX 75201.

4. Defendant does business in Texas, directly or through intermediaries

and offers products or services, including those accused herein of infringement, to

customers and potential customers located in Texas, including in this Judicial

U.S.C. § 271, et seq.

6. Defendant is subject to personal jurisdiction in the State of Texas and

in this Judicial District.

7. Venue for this suit is proper in this Judicial District pursuant to 28

U.S.C. § 1400(b). Defendant is registered to do business in Texas and maintains an

agent for service of process in Texas. On information and belief, Defendant

maintains a place of business within the Western District of Texas, as evidenced by

Defendant has also committed acts of infringement in this Judicial District.

accused products to residents of this Judicial District.

10. Plaintiff’s cause of action arises directly from Defendant’s business

11. On information and belief, Defendant has derived substantial revenues

employees working in and from this Judicial District.

well as by others acting as Defendant’s agents and/or representatives, such that it

would be reasonable for this Court to exercise jurisdiction consistent with

principles underlying the U.S. Constitution and without offending traditional

notions of fair play and substantial justice.

14. On information and belief, Defendant has also established minimum

the Internet, through intermediaries, representatives and/or agents located within

15. On information and belief, Defendant has also purposefully and

On information and belief, Defendant’s customers in the State of Texas, including

and use Defendant’s products.

and continues to intentionally instruct customers and potential customers in Texas

marketed in Texas through regular and established distribution channels; (iv)

has and continues to be part of Defendant’s established distribution channels; (xii)

offend the traditional notions of fair play and substantial justice.

18. Plaintiff has partnered with inventors and entrepreneurs to provide

solutions relates to communication technologies, applicable to multiplayer video

19. Plaintiff’s predecessor-in-interest partnered with Kwok-wai Cheung,

University and a Ph.D. from the California Institute of Technology.

Area of Bell Communications Research from 1987 until 1992.

Information Engineering at the Chinese University of Hong Kong. He served as the

Division Head of the Department of Information Engineering from Aug. 1, 2009

and Technologies, Faculty of Engineering, at the Chinese University of Hong Kong

from May 1996 until September 2001.

22. Dr. Cheung is a named inventor on more than 40 patents, including

more than 20 United States patents.

Citation Index (SCI) citations.

24. Dr. Cheung has been the principal investigator or co-principal

investigator on research grants totaling more than $10,000,000 (U.S.).

27. Defendant is a successful video game developer and publisher. In

Fortnite is played by multiple players concurrently. On information and belief,

hybrid-tower, defense-shooter-survival game in which four players fight

are given complete freedom to create worlds and battle arenas.

28. Fortnite allows players to communicate with one another. Such