US 20220215248A1

IN
( 19 ) United States
( 12 ) Ramirez
PatentetApplication Publication ((4310)) Pub
Pub.. Date
No .: :US 2022/0215248 A1
al . Jul . 7 , 2022
( 54 ) METHOD AND SYSTEM FOR MACHINE Related U.S. Application Data
LEARNING USING A DERIVED MACHINE
LEARNING BLUEPRINT ( 60 ) Provisional application No. 63 / 133,446 , filed on Jan.
4 , 2021 .
( 71 ) Applicant: Covid Cough , Inc. , Greenwood Village , Publication Classification
CO (US ) (51 ) Int . Ci.
( 72 ) Inventors: Maurice A. Ramirez , Lake Wales, FL GO6N 3/08 ( 2006.01 )
(US ) ; Mark Fogarty , Seneca , SC (US ); (52) U.S. CI.
Michael V. Bivins, Orlando, FL (US ); CPC GO6N 3/08 (2013.01 )
Robert Durham , Jacksonville , FL ( 57 ) ABSTRACT
(US ) ; Allison A. Sakara , Lake Wales, Systems and methods of the present disclosure enable signal
FL ( US ) ; Mona Kelley, Springfield , TN data signature detection using a memory unit and processor,
(US ) ; Karl Kelley, Springfield, TN where the memory using stores a computer program or
(US ) ; Morgan Cox , Lakewood, CO computer programs created by the physical interface on a
(US ) ; Nolan Donaldson , Denver, CO temporary basis . The computer program , when executed ,
( US ) ; Adam Stogsdil, Cypress, TX
(US ) ; Simon Kotchou , Phoenix , AZ cause the processor to perform steps to receive a signal data
(US ) ; Robert F. Scordia , Brooklyn , NY signature recording from at least one data source, receive a
(US ) ; Kitty Kolding , Parker, CO (US ); dataset of labeled signal data signature recordings including
Anne Humpich , Downers Grove , IL signal data signature recording labels , identify, using at least
(US ) ; Michelle Archuleta , Lakewood, one machine learning model, boundaries within the dataset
CO (US ) of labeled signal data signature recordings , classify the
signal data signature recording to produce an output label
using a compendium of signal data signature classifiers
( 21 ) Appl. No .: 17 /568,539 based on the boundaries within the dataset of labeled signal
data signature recordings, determine an output type of the
signal data signature recording, and display the output label
( 22 ) Filed : Jan. 4 , 2022 on a display media .

mosa grigie Box System

Sxs osta Sigistusaste Sye : 11

Franko 198o Bros02
Corodos

Du 118

??? ? Steps
Patent Application Publication Jul . 7 , 2022 Sheet 1 of 12 US 2022/0215248 A1

1
***

Signature Detected

FIG . 1A
Patent Application Publication Jul . 7 , 2022 Sheet 2 of 12 US 2022/0215248 A1

Data Barrier - 10

Source Labels Target Labels

118 148

Source Model Transfer Learning Target Model

116 121

1
1
Source Data Target Data
108 138

FIG . 1B
Patent Application Publication Jul . 7 , 2022 Sheet 3 of 12 US 2022/0215248 A1

Statis
Lexus
X.3888

34 ??? ???
nama

FIG , 1C
Patent Application Publication Jul . 7 , 2022 Sheet 4 of 12 US 2022/0215248 A1

goc

??
Fledarmitendg .
FIG
1D

Clenatmrinlg

DQ
Patent Application Publication Jul . 7 , 2022 Sheet 5 of 12 US 2022/0215248 A1

License
Server Org - 1

Org - 2
Server
SN Noda

O
Q
SWO
?
Node
3
SN Node
**

? Org
License

SL Nodes
Org - 3
FIG . 1E
Patent Application Publication Jul . 7 , 2022 Sheet 6 of 12 US 2022/0215248 A1

Swarm edge node **

OF - ] 0
FIG . 1F
Patent Application Publication Jul . 7 , 2022 Sheet 7 of 12 US 2022/0215248 A1

Microphone
200
Wav file 201
V
Signal Data Signature
Pre - processing 202
Symptoms 204
Feature Extraction
Medical History 203
205

Feature vector 206

Machine
Learning
Model ( s )
113

Boundaries 114
Ensemble of
n Classifiers
207
.
FIG . 2
Patent Application Publication Jul . 7 , 2022 Sheet 8 of 12 US 2022/0215248 A1

Ensemble of Classifiers in Parallel Flow Diagram

Baseline
Comparatos
GMM
Classifier

?? ?
305

Pre CNN Deterministic

Phone Processing HMM Classifier 1 Oracle or
301 Filters Detectos Algorithm
302 303 307
CNN
Classifier 2
306

CNN
Classifier 3
?
CNN
Classifier 4
306

Data CNN
Pipeline Classifier 5
304

FIG . 3
Patent Application Publication Jul . 7 , 2022 Sheet 9 of 12 US 2022/0215248 A1

-400

403 404
-402

Client
Device
REC
? ????
Client
Device
Client
Device

Network
405

-407

-406
Network Server
FIG . 4 ( NS )
Network Server
( NS )
Patent Application Publication Jul . 7 , 2022 Sheet 10 of 12 US 2022/0215248 A1

500

{ Cloud Cloud
Service Platform
$ 1

}
Cloud Cloud
Infrastructure Storage
-502
Client
525

5120
-5026
Network
Client 506

W
5126 -504
513
-502a
Client 508 Server Device A Server Device B -514
505 Processor Processor
RAM
-510 Memory Memory -516
Processor 517 Network
Network
Engine Engine
512a

-507 -515
Network Network
Database Database
? B

FIG . 5
Patent Application Publication Jul . 7 , 2022 Sheet 11 of 12 US 2022/0215248 A1

Cloud Cloud
Service Platform
( e.g. ,
> (e.g., Web
Queue ) Frontend )

Cloud
Infrastructure Cloud
( e.g. , VMs ) Storage
(e.g.,
Database )

525

FIG . 6
Patent Application Publication Jul . 7 , 2022 Sheet 12 of 12 US 2022/0215248 A1

Web Browser, Mobile App , Thin Client ,>

Terminal Emulator, ...

704
????????

Aplicaton SaaS
NO

CRM , Email, Virtual Desktop , Communication , Games , ...

706

Paas

Platform Execution Runtime, Database , Web Server,

Development Tools, ...
708

laas

Infra struc e Virtual Machines, Servers, Storage, Load Balancers,

Network , ...
710

FIG . 7
US 2022/0215248 A1 Jul. 7. 2022
1

METHOD AND SYSTEM FOR MACHINE computing and data analytics, has increased the importance
LEARNING USING A DERIVED MACHINE of data , impacting not just information industries, but tra
LEARNING BLUEPRINT ditional industries as well . The use of data analytics in
virtually all industries has increased efficiency, and made the
RELATED APPLICATION ( S ) movement of data more important. Organizations increas
[ 0001 ] This application claims priority to and the benefit ingly rely on data for a number of purposes, including to
ofU.S. Provisional Application No. 63 / 133,446 , filed Jan. 4 , monitor production systems, manage global workforces ,
2021 , which is hereby incorporated by reference in its monitor supply chains , and support products in the field in
entirety . real time . Organizations collect and analyze personal data to
better understand customers ' preferences and willingness to
TECHNICAL FIELD pay , and adapt their products and services accordingly.
[ 0007] Barriers to data flows, such as data - residency
[ 0002 ] The present disclosure relates generally to machine requirements that confine data within a country's borders, a
learning classifiers utilizing - a strategic machine learning as concept known as “ data localization ,” as well as technical
>

a method and system for use of federated data , machine impediments to sharing data exist that provide obstacles to
learning and swarm learning for a derived strategic blueprint efficient implementation of data analytics. Data localization
facilitating machine learning across data boundaries derived can be explicitly required by law or is the de facto result of
blueprint. a culmination of other restrictive policies that make it
unfeasible to transfer data, such as requiring companies to
BACKGROUND store a copy of the data locally, requiring companies to
[ 0003 ] Deep learning approaches have caused tremendous process data locally, and mandating individual or govern
advances in many areas of computer science . Deep learning ment consent for data transfers .
is aa branch of machine learning where the learning process [ 0008 ] Prior solutions are limited by software programs
is done using deep and complex architectures such as that require human input and human decision points, algo
recurrent convolutional artificial neural networks . Many rithms that fail to capture the underlying distribution of
computer science applications have utilized deep learning signal data signature, algorithms that require balanced data
such as computer vision , speech recognition, natural lan sets , algorithms that are brittle and unable to perform well on
guage processing , sentiment analysis, social network analy datasets that were not present during training. Many gov
sis , and robotics . The success of deep learning enabled the ernments place restrictions on the movement of data inter
application of learning models such as reinforcement learn nationally that prior solutions fail to resolve or address.
ing in which the learning process is only done by trial- and SUMMARY OF THE DISCLOSURE
error , solely from actions rewards or punishments. Deep
reinforcement learning come to create systems that can learn [ 0009 ] This specification describes a signal data signature
how to adapt in the real world . As deep learning utilizes deep detection system that includes a machine learning derived
and complex architectures, the learning process usually is strategy for training a compendium of signal data signature
time and effort consuming and need huge labeled data sets . classifiers by applying signal data signature classifiers at the
This inspired the introduction of transfer and multi -task natural boundaries within the dataset (e.g. , underlying fea
learning approaches to better exploit the available data tures that lead to class distinctions ). The signal data signa
during training and adapt previously learned knowledge to ture detection system components include input data , com
emerging domains, tasks, or applications. puter hardware, computer so ware , and output data that can
[ 0004 ] Traditional deep learning based approaches have be viewed by a hardware display media or paper. A hardware
been applied to develop classifiers for a number of respira display media may include a hardware display screen on a
tory illnesses using cough signal data signature recordings. device ( computer, tablet , mobile phone ) , projector, and other
The challenge with deep learning models that are very types of display media .
specialized to a particular domain or even a specific task is [ 0010 ] Signal Data Signature detection, characterization
that they are unable to differentiate or further classify and classification is the task of recognizing a source signal
negatives . There becomes an uncertainty about whether data signature and its respective temporal parameters within
there is certain degree of statistical luck as opposed to a source signal data stream or recording . Sound Event
further discrimination and classification of the negative Detection ( SED ) is an example of signal data signature
category . Deep learning models can be trained for classifi detection with many different applications. SED is the task
cation and prediction tasks , however they are constrained by of recognizing sound events and their respective temporal
sample imbalance . In order for a deep neural network to be start and end time in an audio recording. SED aims at
predictive across multiple applications it must be given a processing the continuous acoustic signal and converting it
balanced set of labeled signal data . into symbolic descriptions of the corresponding sound
[ 0005 ] The free flow of data across borders is essential for events as well as the timing of those events . SED and other
the digital economy, yet many governments place restric signal data signature detection algorithms may include con
tions on the movement of data internationally. Cross -border text - based indexing, retrieval in multimedia databases ,
flows of data are currently regulated by a number of inter unobtrusive monitoring in health care , surveillance, and
national, regional and national instruments and laws medical diagnostics.
intended to protect individuals ' privacy, the local economy [ 0011 ] The application of signal data signature detection
or national security. Other data barriers may exist , as well , as a medical diagnostic or screening tool is particularly
such as bandwidth and networking limitations , etc. attractive as it represents a non - intrusive, real- time diagnos
[ 0006 ] The increased digitalization of organizations, tic that can be essential during public health crisis . Public
driven by the rapid adoption of technologies such as cloud health situations may be exacerbated by the lack of real - time
US 2022/0215248 A1 Jul. 7. 2022
2

testing diagnostics which in turn compromises the safety of receive a dataset of labeled signal data signature recordings
vulnerable populations. Further, the ability to identify a including signal data signature recording labels ; where the
signal data signature diagnostic of a particular condition or memory unit is configured to store the signal data signature
disease can have significant benefits for limiting the spread recording and dataset of labeled signal data signature record
of and recovery from an infectious disease . ings created by the physical interface on a temporary basis ;
[ 0012 ] Generally, the system may perform signal data identify, using at least one machine learning model , bound
signature detection on a signal data signature recording aries within the dataset of labeled signal data signature
using a compendium of signal data signature classifiers that recordings; classify the signal data signature recording to
have been trained using a ML - derived blueprint for signal produce an output label using a compendium of signal data
data signature classifiers using paired signal data signature signature classifiers based on the boundaries within the
and respiratory condition dataset . The signal data signature dataset of labeled signal data signature recordings ; deter
detection system receives input paired signal data signature mine an output type of the signal data signature recording ;
data and a corresponding label that indicates the presence or and display the output label on a display media .
absence of a medical condition . The signal data signature
detection system includes of computer hardware that when BRIEF DESCRIPTION OF THE DRAWINGS
executed by a processor performs the following steps : 1 ) [ 0016 ] FIG . 1A , FIG . 1B , FIG . 1C , FIG . 1D , FIG . 1E and
splits the paired signal data signature dataset into a training, FIG . 1F illustrate a signal data signature detection system in
testing , and validation datasets; 2 ) defines the model defines accordance with aspects of embodiments of the present
unique class boundaries for each class within the paired disclosure .
training signal data signature dataset ; 3 ) utilizes the natural [ 0017] FIG . 2 illustrates a machine learning derived
boundaries within the paired training signal data signature boundaries in accordance with aspects of embodiments of
dataset to define a source and target models such that the the present disclosure.
source model will be developed with the entire training [ 0018 ] FIG . 3 illustrates a signal data signature classifier
dataset and the target models will be developed with subsets system in accordance with aspects of embodiments of the
of the paired signal data signature training dataset; 4 ) signal present disclosure.
data signature classifier techniques such as feature extrac
tors , weight -adjustment, and tuning layers will be applied to [ 0019 ] FIG . 4 depicts a block diagram of an exemplary
the target models; 5 ) target models and source model will be computer -based system and platform 400 in accordance with
tuned using the paired testing signal data signature dataset; one or more embodiments of the present disclosure .
6 ) the target models and source model will be used as a [ 0020 ] FIG . 5 depicts a block diagram of another exem
compendium of signal data signature classifiers on the plary computer -based system and platform 500 in accor
unseen paired signal data signature testing dataset . The dance with one or more embodiments of the present disclo
signal data signature detection system includes of input data sure .
paired signal data signature recording data with aa label and [ 0021 ] FIG . 6 illustrates schematics of exemplary imple
computer hardware that when executed by a processor mentations of the cloud computing /architecture ( s) in which
returns a compendium of signal data signature classifiers , the exemplary inventive computer - based systems/ platforms,
such that when the signal data signature detection system the exemplary inventive computer -based devices , and / or the
receives another signal data signature recording without a exemplary inventive computer -based components of the
label the signal data signature detection system will return present disclosure may be specifically configured to operate .
an output label that can be viewed by a hardware display [ 0022 ] FIG . 7 illustrates schematics of exemplary imple
media or paper. mentations of the cloud computing /architecture ( s) in which
[ 0013 ] Advantages of the signal data signature detection the exemplary inventive computer -based systems/platforms,
system are the following 1 ) can generate a compendium of the exemplary inventive computer -based devices , and / or the
signal data signature classifiers from data, 2 ) can generate a exemplary inventive computer -based components of the
compendium of signal data signature classifiers that can be present disclosure may be specifically configured to operate .
used to predict a label from an unlabeled signal data signa [0023]
ture recording, 3 ) generates signal data signature classifiers
that can be used to diagnose acute and / or chronic conditions .
[ 0014 ] The details of one or more embodiments of the Drawings - - - Reference Numerals
subject matter of this specification are set forth in the 100 Signal data signature 101 Signal data signature
accompanying drawings and the description below. Other Detection System Recording
features , aspects , and advantages of the subject matter will 102 Hardware
104 Memory
103 Computer
become apparent from the description , the drawings, and the 106 Network Controller
105 Processor
107 Network
claims . 108 Data Sources 109 Software
[ 0015 ] Systems and methods of illustrative embodiments 110 Signal data signature 111 Signal data signature
of the present disclosure include at least one hardware Detector System Classifier System
112 Signal data signature 113 ML models
device including a processor and a memory unit, where the classifier System
memory unit is configured to store a computer program or 114 Boundaries 121 Target model ( s )
computer programs created by the physical interface on a 116 Source model 117 Knowledge transfer
temporary basis . The computer program , when executed , 118 Signal data signature & Signal data 119 Display Screen
signature Type
causes the processor to perform steps to : receive a signal 120 Paper 121 Compendium of Signal data
data signature recording from at least one data source ; where Signature
the memory unit is configured to store the data sources Classifiers
created by the physical interface on a temporary basis ;
US 2022/0215248 A1 Jul. 7. 2022
3

-continued Deep Learning across data borders using Transfer Learning.

In some embodiments, the technical solution may be accom
Drawings - - - Reference Numerals plished with a signal data signature detection system that
200 Microphone 201 .Wav file includes of hardware devices (e.g. , desktop, laptop , servers ,
202 Signal data signature 203 Feature Extraction tablet , mobile phones, etc. ), storage devices (e.g. , hard drive
Pre - processing disk, floppy disk, compact disk ( CD ) , secure digital card ,
204 Symptoms 205 Medical History solid state drive, cloud storage , etc. ), delivery devices (pa
206 Feature Vector 207 Ensemble of n Classifiers
300 Task 1 , T1 301 Task 2 , T2 per, electronic display ), a computer program or plurality of
302 Task 3 , T3 303 Source Data computer programs, and a processor or plurality of proces
304 Source Model 305 Target Data sors . A signal data signature detection system when executed
306 Target Labels on a processor (e.g. , CPU , GPU) would be able to identify
a specific signal data signature from other types of signal
data signatures and delivered to clinicians and /or end -users
DETAILED DESCRIPTION through a delivery device ( paper, electronic display ). The
model features derived from the signal data signatures flow
Signal Data Signature Detection System across data boundaries using transfer learning.
[ 0024 ] FIGS . 1 through 7 illustrate systems and methods [ 0028 ] The free and efficient flow of data allows machine
of signal data signature detection and machine learning learning models and other data analytics solutions to access
model training. The following embodiments provide tech the global range and quality of services, and permits such
nical solutions and / or technical improvements that over data analytics solutions to more efficiently leverage the
come technical problems, drawbacks and /or deficiencies in analysis from across data barriers while overcoming tech
the technical fields involving model training and machine nical hurdles with accessing data . For example:
learning techniques for efficient use of data in the presence [ 0029 ] Services can emerge and be adopted in one
of data barriers. As explained in more detail, below , tech national market, then expand readily to other markets,
nical solutions and / or technical improvements herein bringing benefits for consumers and businesses .
include aspects of improved machine learning model train [ 0030 ] Startup businesses can have a global reach from
ing utilizing - federated data , machine learning and swarm Day One by establishing an internet presence that is
learning for a derived strategic blueprint facilitating machine simultaneously national and international.
learning across data boundaries. Based on such technical [ 0031 ] Internet infrastructure suppliers , such as cloud
features, further technical benefits become available to users computing providers and mobile operators, can struc
and operators of these systems and methods . Moreover, ture their services to serve large numbers of clients in
various practical applications of the disclosed technology multiple markets without exporting local data .
are also described, which provide further practical benefits [ 0032 ] Software platforms can scale up ( and down ) in
to users and operators that are also new and useful improve development , making direct or indirect use of cloud
ments in the art. and Software -as -Medical -Device ( SaMD ) providers.
[ 0025 ] The present disclosure relates generally to machine [ 0033 ] In some embodiments , a Data Management
learning classifiers . Embodiments of the present disclosure (MDM) architectural model may help bridge a gap among
include signal data signature detection , signal data signature organizations, technologies, and users that results from data
classification , utilizing - a strategic machine learning as a barriers. Enterprise Data Management, an IT discipline, is
method and system for use of federated data, machine composed of a set of tools and processes to define enterprise
learning and swarm learning for aa derived strategic blueprint data entities of an organization . Enterprise data management
facilitating machine learning across data boundaries. In objectives are to organize and manage the organization's
some embodiments, the derived strategic blueprint is formed enterprise data . In some embodiments , the MDM may
from a compendium of signal data signature classifiers from include an architectural type including, e.g. , Centralized,
training data whereby a signal data signature classifier is Federated or a combination thereof.
used based on the natural decision boundaries within the
signal data signature that exchange data across data bound [ 0034 ] In some embodiments, in a Centralized Data Model
aries by using deep learning, transfer learning to exchange (CDM ) , data may be consolidated in one repository. Using
model features and Swarm learning to disseminate these CDM may resolve data duplications, inconsistent master
model features to multiple instances of the same AI /ML data , and improve data quality. However, implementing
ensembles . CDM may require users to overcome challenges such as
[ 0026 ] Many sources for data boundaries that impede the crossing data barriers, geographical locations of the appli
use of data to train machine learning models may exist . For cations, cost of the implementation, and compliance with
example, in data localization, countries impose requirements privacy rules and regulations.
for organizations to use local data storage or technology, [ 0035 ] In some embodiments , a Federated Data Model
which prevents communicating the data beyond a particular (FDM) may enable an organization to extend data and
locale and create unnecessary duplication and cost . The use business services to inquire data from multiple sources . An
of transfer learning for cross border flow of model features FDM may make data available to all users and / or partners of
without cross border data flow provides a solution and an organization. Yet, implementing FDM comes with many
principle that mitigate these risks without restricting the challenges such as data barriers, synchronization of data
benefits of machine learning. between transactional and master data , network connectivity
[ 0027] In some embodiments, a technical solution may between the sources and MDM hub, privacy rules and
include to classify and tag signal data signatures from regulations, performance , maintenance, and identifying
datasets then flow the model features derived by AI /ML roles and responsibilities.
US 2022/0215248 A1 Jul. 7. 2022
4

[ 0036 ] In some embodiments, AI/ML model features by a processor ( s ) with an unlabeled data source ( s) generates
derived from Centralized Data and / or Federated Data is not an output label ( s ) (the output 118 ) which is shown on a
subject to cross border data flow restrictions because the hardware device such as a display screen or sent to a
AI /ML model features are the product of the AI/ ML Deep hardware device such as a printer where it manifests as
Learning processing of the source data and no longer physical printed paper that indicates the diagnosis of the
contains the source data itself. Similarly, the AI /ML model input signal data recording and signal data signature .
features contain no personal data or identifiers and therefore [ 0042 ] In some embodiments, the data sources 108 that are
is not subject to cross border data flow data privacy rules or retrieved by a hardware device 102 in one of other possible
regulations . Additionally, the movement of the AI /ML embodiments includes for example but not limited to : 1 )
model features may provide network and storage efficiencies imbalanced paired training dataset of signal data signature
that would not be possible with transferring training data , recordings and labels and unlabeled signal data signature
which may include much larger quantities of data . recording, 2 ) balanced paired training dataset of signal data
[ 0037] Determining which architectural model is suitable signature recordings and labels and unlabeled signal data
for a particular platform depends on several factors; includ signature recording, 3 ) imbalanced paired training dataset of
ing use of the platform data , number of applications (do video recordings and labels and unlabeled video recording,
mains ) that will use the master data, derivation of model 4 ) imbalanced paired training dataset of video recordings
features, cross border data flow rules and regulations , devel and labels and unlabeled signal data signature recording, 5 )
opment and availability costs , delivery schedule, perfor paired training dataset of signal data signature recordings
mance , efficiency, limitations , risk , training, operations, and labels and unlabeled video recording . In some embodi
compliances, deployment, security, accessibility, depend ments, a “balanced ” training dataset may include an equal
ability, data quality, stability , maintainability, reliability , number of training signal data signature records for each
availability, flexibility, scalability, predictability and cross classification, such as equal numbers of training data for
border data privacy rules and regulations . each of a first classification and for ?a second classification in
[ 0038 ] FIG . 1A illustrates a signal data signature detection a binary classification , such as , e.g. , a positive and a negative
system 100 with the following components: input 101 , classification in a diagnosis classification . In some embodi
hardware 102 , software 109 , and output 118. The input is a ments, an “ imbalanced ” training dataset may include an
signal data signature recording such as a signal data signa unequal number of training signal data signature records for
ture recording captured by a sensor, a signal data signature a first classification and for a second classification in a
recording captured on a mobile device , and a signal data binary classification, such as , e.g. , a positive and a negative
signature recording captured on any other device , among classification in a diagnosis classification . Example ratios
others. The input 101 may be provided by an individual, for an imbalanced training dataset may include , e.g. , 70:30,
individuals or a system and recorded by a hardware device 50:25:25 , 60:40 , 60:20:20 , or any other suitable ratio . Such
102 such as a computer 103 with a memory 104 , processor a training scheme influences the training, machine learning
105 and or network controller 106. A hardware device is able and probability predictions of the classifiers trained with the
to access data sources 108 via internal storage or through the balanced and /or unbalanced SDS data sets . Unbalanced sets
network controller 106 , which connects to a network 107 . tend to bias the ML towards the higher ratio SDS as a
[ 0039 ] In some embodiments, the signal data signature prediction where balanced sets tend to bias towards more
detection system 100 may identify a classification label that equal probabilities .
indicates the presence or absence of a disease when the [ 0043 ] In some embodiments, the data sources 108 and the
system is provided with unbalanced paired signal data signal data signature recording input 101 are stored in
signature recordings and their corresponding disease labels memory or a memory unit 104 and passed to a software 109
and another unlabeled signal data signature recording. These such as computer program or computer programs that
embodiments are advantageous for identifying classification executes the instruction set on a processor 105. The software
labels such as , e.g. , underlying respiratory illnesses for 109 being a computer program executes a signal data
providing in -home, easy to use diagnostics for respiratory signature detector system 110 and a signal data signature
conditions , such as , e.g. , COVID - 19 , bronchitis , pneumonia, classification system 111. The signal data signature classi
among others or any combination thereof. fication system 111 executes a signal data signature classifier
[ 0040 ] In some embodiments, in order to achieve a soft system 112 on a processor 105 such that the paired training
ware program that is able , either fully or partially, to detect dataset is used to train machine learning ( ML ) models 113
and diagnose signal data signatures, that program generates that generate boundaries within the dataset 114 whereby the
a compendium of signal data signature classifiers 121 from boundaries inform the scope and datasets of target model ( s )
a training dataset. Another challenge is that such a program 121 and the source model 116 , such that knowledge is
must be able to scale and process large datasets . transferred 117 from the source model 116 to the target
[ 0041 ] Embodiments of the present disclosure are directed model ( s) 121 .
to the signal data signature detection system 100 whereby a [ 0044 ] In some embodiments, the boundaries may include
signal data recording ( the input 101 ) is provided by an thresholds set for determination of a diagnosis based on the
individual or individuals (s) or system into a computer hard classifier predictions. For example, if the predictions from
ware whereby labeled data sources and unlabeled data the classifier span 0.001 (not COVID ) to 0.999 (IS COVID )
source ( s ) are stored on a storage medium and then the then thresholds (boundaries) are used to determine the lower
labeled data sources and unlabeled data source (s ) are used as limit for IS COVID prediction values , such as , 0.689 , above
input to a computer program or computer programs which which the diagnosis is COVID . While a NOT COVID
when executed by a processor(s ) provides compendium of prediction value threshold ( boundary ), say 0.355 defines the
signal data signature classifiers 121 saved to a hardware limit below which the diagnosis is no COVID disease .
device as executable source code such that when executed Between the boundaries (0.3551 to 0.6889 ) is indeterminant.
US 2022/0215248 A1 Jul. 7. 2022
5

In some embodiments , the thresholds may be learned via the the components described herein may include computer
training of the ML models 113 , experimentally determined , hardware and /or executable software which is stored on a
or determined by any other suitable technique. The positive computer -readable medium for execution on appropriate
diagnosis boundary may include , e.g. , between 0.400 and computing hardware. The terms “ computer -readable
0.499 , between 0.500 and 0.599 , between 0.600 and 0.699 , medium ” or “ machine readable medium ” should be taken to
between 0.700 and 0.799 , between 0.800 and 0.899 , between include a single medium or multiple media that store one or
0.900 and 0.999 , for example 0.680 , 0.681 , 0.682 , 0.683 ,
2 more sets of instructions. The terms “ computer -readable
0.684 , 0.685 , 0.686 , 0.687 , 0.688 , 0.689 , 0.690 , 0.691 , medium ” or “ machine readable medium ” shall also be taken
0.692 , 0.693 , 0.694 , 0.695 , 0.696 , 0.697 , 0.698 , 0.699 , to include, but not be limited to , solid - state memories, and
0.700 , etc. The negative diagnosis boundary may include, optical and magnetic media . For example , " computer -read
e.g. , between 0.100 and 0.199 , between 0.200 and 0.299 , able medium ” or “ machine readable medium ” may include
between 0.300 and 0.399 , between 0.400 and 0.499 , for Compact Disc Read -Only Memory ( CD - ROMs), Read -Only
example 0.350 , 0.351 , 0.352 , 0.353 , 0.354 , 0.355 , 0.356 , Memory (ROMs ) , Random Access Memory (RAM ), and / or
0.357 , 0.358 , 0.359 , 0.360 , 0.361 , 0.362 , 0.363 , 0.364 , Erasable Programmable Read -Only Memory (EPROM ).
0.365 , 0.366 , 0.367,0.368 , 0.369,0.370 , etc. The signal data The terms " computer-readable medium ” or “ machine read
signature classifier system 112 defines the boundaries and able medium ” shall also be taken to include any non
scope of target model ( s ) 121 and source model 116 whereby transitory storage medium that is capable of storing , encod
knowledge is transferred 117 from the source model 116 that ing or carrying a set of instructions for execution by a
has been trained on a larger training dataset to the target machine and that cause a machine to perform any one or
model ( s ) 121 that are trained on a smaller training dataset. more of the methodologies described herein . In other
In some embodiments, the output 118 is aa label that indicates embodiments, some of these operations might be performed
.

the presence or absences of a condition given that an by specific hardware components that contain hardwired
unlabeled signal data signature recording is provided as logic . Those operations might alternatively be performed by
input 101 to the signal data signature detection system such any combination of programmable computer components
that the output 118 can be viewed by a reader on a display and fixed hardware circuit components.
screen 119 or printed on paper 120 . [ 0047] In one or more embodiments of the signal data
[ 0045 ] In some embodiments, the signal data signature signature classifier system 111 software 109 includes the
detection system 100 hardware 102 includes the computer signal data signature classifier system 112 which will be
103 connected to the network 107. The computer 103 is described in detail in the following section .
configured with one or more processors 105, a memory or [ 0048 ] In one or more embodiments of the signal data
memory unit 104 , and one or more network controllers 106 . signature detection system 100 the output 118 includes a
In some embodiments, the components of the computer 103 strongly labeled signal data signature recording and identi
are configured and connected in such a way as to be fication of signal data signature type. An example would be
operational so that an operating system and application signal data signature sample from a patient which would
programs may reside in a memory or memory unit 104 and include: 1 ) aa label of the identified signal data signature type,
may be executed by the processor or processors 105 and data 2 ) or flag that tells the user that a signal data signature was
may be transmitted or received via the network controller not detected . The output 118 of signal data signature type or
106 according to instructions executed by the processor or message that a signal data signature was not detected will be
processor ( s) 105. In some embodiments, a data source 108 delivered to an end user via a display medium such as but not
may be connected directly to the computer 103 and acces limited to a display screen 119 (e.g. , tablet, mobile phone,
sible to the processor 105 , for example in the case of a signal computer screen ) and / or paper 120 .
data signature sensor, imaging sensor, or the like . In some [ 0049 ] In some embodiments, the label produced by the
embodiments , a data source 108 may be executed by the signal data signature classifier system 111 may include a
processor or processor ( s ) 105 and data may be transmitted or start time , an end time or both of a segment an audio
received via the network controller 106 according to instruc recording of the input 101. In some embodiments, the signal
tions executed by the processor or processors 105. In one data signature classifier system 111 may be trained to
embodiment, a data source 108 may be connected to the identify a modified audio recording in the signal data
signal data signature classifier system 112 remotely via the signature recording 101 based on a matching to a target
network 107 , for example in the case of media data obtained distribution . In some embodiments, the modified signal data
from the Internet. The configuration of the computer 103 signature recording may include a processing that extracts
may be that the one or more processors 105 , memory 104 , segments of the audio recording. For example , the signal
or network controllers 106 may physically reside on mul data signature classifier system 111 may identify, e.g. , indi
tiple physical components within the computer 103 or may vidual coughs in a recording of multiple coughs, and extract
be integrated into fewer physical components within the a segment for each cough having a start time label at a
computer 103 , without departing from the scope of the beginning of each cough and an end time label at an end of
present disclosure. In one embodiment, a plurality of com each cough . In some embodiments, the audio recording may
puters 103 may be configured to execute some or all of the be a single cough , and the signal data signature classifier
steps listed herein , such that the cumulative steps executed system 111 may label the start time and the end time of the
by the plurality of computers are in accordance with the single cough to extract the segment of the audio recording
present disclosure. having the cough .
[ 0046 ] In some embodiments, a physical interface is pro Signal Data Signature Classifier System
vided for embodiments described in this specification and
includes computer hardware and display hardware (e.g. , the [ 0050 ] In some embodiments , a signal data signature
display screen of aa mobile device ) . In some embodiments , classifier system 112 with real- time training of machine
US 2022/0215248 A1 Jul. 7. 2022
6

learning models 113 and the real - time training of model ( s ) a threshold at which the node is activated . In some embodi
121 and the source model 116 , hardware 102 , software 109 , ments and, optionally, in combination of any embodiment
and output 118. FIG . 2. illustrates an input to the signal data described above or below, the exemplary aggregation func
signature classifier system 112 that may include but is not tion may be a mathematical function that combines ( e.g. ,
limited to paired training dataset of signal data signature sum , product, etc. ) input signals to the node. In some
recordings and corresponding signal data signature labels embodiments and, optionally , in combination of any
and an unpaired signal data signature recording 101 that is embodiment described above or below , an output of the
first received and processed as a signal data signature wave exemplary aggregation function may be used as input to the
by a hardware device such as a microphone 200. In addition, exemplary activation function . In some embodiments and,
the signal data signature labels may be input into the signal optionally, in combination of any embodiment described
data signature classifier system using a physical hardware above or below, the bias may be a constant value or function
device such as a keyboard. that may be used by the aggregation function and / or the
[ 0051 ] In some embodiments, the signal data signature activation function to make the node more or less likely to
classifier system 112 uses a hardware 102 , which includes of
2 be activated .
a memory or memory unit 104 , and processor 105 such that [ 0060 ] In some embodiments, training the set of signal
software 109 , a computer program or computer programs is data signature classifiers may include transfer learning to
executed on a processor 105 and trains in real- time a set of share model features amongst the signal data signature
signal data signature classifiers. The output from signal data classifiers in the set of signal data signature classifiers . In
signature classifier system 112 is a label 118 that matches some embodiments, the model features may include , e.g. ,
and diagnosis a signal data signature recording file . A user Fast Formant Transform spectrogram , MEL spectrogram ,
is able to the signal data signature type output 118 on a MFCC Spectrogram , as well as specific spectrum features
display screen 119 or printed paper 120 . such as formant configuration or formant slurring, among
[ 0052 ] In some embodiments, the signal data signature other features or any combination thereof.
classifier system 112 may be configured to utilize one or [ 0061 ] As illustrated FIG . 1B , a source model including
more exemplary Al/machine learning techniques chosen the signal data signature classifier model 116 may output
from , but not limited to , decision trees, boosting , support source labels 118 based on a large set of source data 108 .
vector machines , neural networks, nearest neighbor algo Through transfer learning, learned features of the source
rithms, Naive Bayes, bagging , random forests, and the like . model 116 may be provided across a data barrier 10 to a
In some embodiments and , optionally, in combination of any target model in the compendium of source signal data
embodiment described above or below, an exemplary neu classifiers 121. As a result, the target model 121 may output
tral network technique may be one of, without limitation , target labels 148 based on the transfer learning using the
feedforward neural network , radial basis function network , learned features and a smaller set of local target data 138 ,
recurrent neural network , convolutional network ( e.g. , thus preserving privacy , confidentiality, network resources
U - net ) or other suitable network . In some embodiments and , and storage resources while transferring model learned
optionally, in combination of any embodiment described across a data barrier 10. In some embodiments, the transfer
above or below , an exemplary implementation of Neural learning of learned features may include aa transfer of inter
Network may be executed as follows: nal neural network connections rather than the data used to
[ 0053 ] a . define Neural Network architecture /model, train the connections, thus preserving the privacy of the data
[ 0054 ] b . transfer the input data to the exemplary neural and complying with , e.g. , data boundaries and other data
network model , barriers.
[ 0055 ] c . train the exemplary model incrementally, [ 0062 ] In some embodiments, there may be different trans
[ 0056 ] d . determine the accuracy for a specific number fer learning strategies and techniques, which can be applied
of timesteps, based on the domain, task at hand, and the availability of
[ 0057 ] e . apply the exemplary trained model to process data . Thus, transfer learning methods can be categorized
the newly - received input data , based on the type of traditional ML algorithms involved,
[ 0058 ] f . optionally and in parallel, continue to train the such as :
exemplary trained model with a predetermined period [ 0063 ] Inductive Transfer learning: In this scenario , the
icity . source and target domains are the same , yet the source
[ 0059 ] In some embodiments and, optionally, in combi and target tasks are different from each other. The
nation of any embodiment described above or below, the algorithms try to utilize the inductive biases of the
exemplary trained neural network model may specify a source domain to help improve the target task . Depend
neural network by at least a neural network topology, a ing upon whether the source domain contains labeled
series of activation functions, and connection weights. For data or not , this can be further divided into two sub
example, the topology of a neural network may include a categories , similar to multitask learning and self - taught
configuration of nodes of the neural network and connec learning, respectively .
tions between such nodes . In some embodiments and, [ 0064 ] Unsupervised Transfer Learning: This setting is
optionally, in combination of any embodiment described similar to inductive transfer itself, with a focus on
above or below , the exemplary trained neural network model unsupervised tasks in the target domain . The source and
may also be specified to include other parameters, including target domains are similar, but the tasks are different. In
but not limited to , bias values / functions and / or aggregation this scenario , labeled data is unavailable in either of the
functions. For example, an activation function of a node may domains.
be a step function , sine function , continuous or piecewise [ 0065 ] Transductive Transfer Learning: In this scenario ,
linear function, sigmoid function , hyperbolic tangent func there are similarities between the source and target
tion, or other type of mathematical function that represents tasks , but the corresponding domains are different. In
US 2022/0215248 A1 Jul. 7. 2022
7

this setting, the source domain has a lot of labeled data , [ 0072 ] These strategies are general approaches which can
while the target domain has none . This can be further be applied towards machine learning techniques. In some
classified into subcategories, referring to settings where embodiments, transfer learning may be applied in the con
either the feature spaces are different or the marginal text of deep learning models , which may represent inductive
probabilities. learning. In some embodiments, the objective for inductive
[ 0066 ] The three transfer categories discussed in the pre learning algorithms is to infer a mapping from a set of
vious section outline different settings where transfer learn training examples. For instance , in cases of classification ,
ing can be applied, and studied in detail. To answer the such as signal data classification , the model learns mapping
question of what to transfer across these categories , some of between input features and class labels . In order for such a
the following approaches can be applied : learner to generalize well on unseen data , its algorithm
[ 0067 ] Instance transfer : In some embodiments, one or works with a set of assumptions related to the distribution of
more signal data signature classifiers may reuse knowl the training data . These sets of assumptions are known as
edge from a source domain for a target task . In some inductive bias . The inductive bias or assumptions can be
embodiments, “ knowledge ” of the classifier may characterized by multiple factors, such as the hypothesis
include, e.g. , internal neural network connections that space it restricts to and the search process through the
compose and define the hidden layers of the classifiers. hypothesis space . Thus, these biases impact how and what is
Thus, the instance transfer may include a transfer of learned by the model on the given task and domain .
internal neural network connections. In most cases , the [ 0073 ] In some embodiments, inductive transfer tech
source domain data cannot be reused directly. Rather, niques may utilize the inductive biases of the source task to
there are certain instances from the source domain that assist the target task , such as by adjusting the inductive bias
can be reused along with target data to improve results . of the target task by limiting the model space , narrowing
In case of inductive transfer, modifications such as down the hypothesis space , or making adjustments to the
AdaBoost , which may help utilize training instances search process itself with the help of knowledge from the
from the source domain for improvements in the target source task . In some embodiments, inductive - learning algo
task . rithms may also utilize Bayesian and Hierarchical transfer
[ 0068 ] Feature-representation transfer: In some techniques to assist with improvements in the learning and
embodiments, feature representation transfer may performance of the target task .
minimize domain divergence and reduce error rates by [ 0074 ] Deep learning has made considerable progress in
identifying “ good” feature representations that can be recent years. This has enabled us to tackle complex prob
utilized from the source domain to target domains , lems and yield amazing results. However, the training time
where “ good ” refers to a signal data signature charac and the amount of data required for such deep learning
teristics that are specific and / or unique to a specific systems are much more than that of traditional ML systems.
disease state and / or health state . Depending upon the Accordingly, in some embodiments, one or more pre -trained
availability of labeled data , supervised or unsupervised deep learning networks with state -of -the- art performance
methods may be applied for feature -representation that have been developed and tested across domains may
based transfers. form the basis of transfer learning in the context of deep
[ 0069 ] Parameter transfer: This approach works on the learning, or deep transfer learning. In some embodiments ,
assumption that the models for related tasks share some the sound signal data classifiers may thus take advantage of
parameters or prior distribution of hyperparameters. the cross -domain deep learning network ( s) via transfer
Unlike multitask learning, where both the source and learning. The transfer learning process can leverage the
target tasks are learned simultaneously, for transfer training of the pre - trained deep learning network across a
learning, we may apply additional weightage to the loss data barrier provide training for the sound signal data
of the target domain to improve overall performance. classifiers without the need for large training data sets.
[ 0070 ] Relational-knowledge transfer: Unlike the pre [ 0075 ] Al -based solutions rely intrinsically on appropriate
ceding three approaches, the relational -knowledge algorithms, but even more so on large training datasets . As
transfer attempts to handle non - IID data , such as data medicine is inherently decentral, the volume of local data is
that is not independent and identically distributed . In often insufficient to train reliable classifiers . As a conse
other words , data, where each data point has a rela quence , centralization of data is one model that has been
tionship with other data points; for instance , social used to address the local limitations . While beneficial from
network data utilizes relational-knowledge - transfer an Al perspective , centralized solutions have inherent dis
techniques. advantages, including increased data traffic and concerns
[ 0071 ] FIG . 1C , FIG . 1D and the following table summa about data ownership , confidentiality, privacy, security and
rizes the relationship between different transfer learning the creation of data monopolies that favor data aggregators .
strategies and what to transfer. Consequently, solutions to the challenges of central AI
models must be effective , accurate and efficient; must pre
Transductive Unsupervised
serve confidentiality, privacy and ethics; and must be secure
Inductive
Transfer Transfer Transfer
and fault -tolerant by design. Federated AI addresses some of
Learning Learning Learning these aspects . Data are kept locally and local confidentiality
issues are addressed, but model parameters are still handled
Instance - transfer
Feature - representation -transfer
Parameter -transfer
? by central custodians , which concentrates power.
[ 0076 ] Furthermore , such star - shaped architectures
Relational-knowledge -transfer decrease fault tolerance . In some embodiments, partially
and / or completely decentralized Al solutions may overcome
current shortcomings, and accommodate inherently decen
US 2022/0215248 A1 Jul. 7. 2022
8

tral data structures and data privacy and security regulations [ 0088 ] Swarm Network ( SN) nodes—In some embodi
in medicine . In some embodiments, integration of the signal ments, these nodes form the blockchain network . In
data signature classifier system in a federated learning some embodiments , the Swarm Network nodes interact
architecture may : with each other using the blockchain platform to main
[ 0077 ] 1 ) keep large medical data locally with the data tain global state information about the model that is
owner; being trained and to track progress ( note that only
[ 0078 ] 2 ) require no exchange of raw data, thereby also metadata is written to the blockchain . The model itself
reducing data traffic and preventing cross border data is not stored in the blockchain .) In some embodiments ,
flow : tSwarm Network nodes use this state and progress
[ 0079 ] 3 ) provide high - level data security ; information to coordinate the working of the Swarm
[ 0080 ] 4 ) guarantee secure , transparent and fair Learning nodes . Each Swarm Learning node registers
onboarding of decentral members of the network with itself with a Swarm Network node as a part of its
out the need for a central custodian ; startup and initialization.
[ 0081 ] 5 ) allow parameter merging with equal rights for [ 0089 ] Sentinel node : In some embodiments, the is a
all members ; and special Swarm Network node. The Sentinel node is
responsible for initializing the blockchain network and
[ 0082 ] 6 ) protect machine learning models from attacks . may be the first node to start.
[ 0083 ] In some embodiments, the federated learning archi [ 0090 ] Swarm Learning Command Interface node
tecture of the signal data signature classifier system 111 may ( SWCI )—In some embodiments, the SWCI node is the
include Swarm Learning ( SL ) , which combines decentral command interface tool to the Swarm Learning frame
ized hardware infrastructures, distributed machine learning work . In some embodiments , the SWCI node may be
based on standardized AI engines with a permissioned 2

used to view the status, control and manage the swarm

blockchain to securely onboard members, to dynamically learning framework . In some embodiments, the SWCI
elect the leader among members, and to merge model may use a secure link to connect to the Swarm Network
parameters. Computation is orchestrated by an SL library node, using the application programming interface
( SLL ) and an iterative AI learning procedure that uses ( API) port. In some embodiments, the SWCI node can
decentral data (Supplementary Information ). connect to any of the SN nodes in a given Swarm
[ 0084 ] FIG . 1E and FIG . 1F illustrates a Swarm Learning Learning framework to manage the framework .
architecture for cross -barrier learning in accordance with [ 0091 ] Server nodesIn some embodiments, the server
one or more embodiments of the present disclosure . nodes provide the security for the whole network . In
[ 0085 ] In some embodiments, Swarm Learning is a decen some embodiments , the platform may run one or more
tralized, privacy - preserving Machine Learning framework . Server nodes that are connected together to form a
This framework utilizes the computing power at , or near, the federation . In some embodiments , the platform
distributed data sources to run the Machine Learning algo includes an Agent Workload Attestor plugin ( not shown
rithms that train the models . It uses the security of a in the figure ) that communicates with the Servers to
blockchain platform to share learnings with peers in a safe attest the identities of the Swarm Network and Swarm
and secure manner . In Swarm Learning, training of the Learning nodes , acquire and manage a Verifiable Iden
model occurs at the edge , where data is most recent, and tity Document (VID ) .
where prompt, data - driven decisions are mostly necessary . [ 0092 ] License Server node_In some embodiments,
In this completely decentralized architecture, only the the license to run the Swarm Learning platform is
insights learned are shared with the collaborating ML peers , installed and managed by the License Server node .
not the raw data . This tremendously enhances data security
and privacy. In FIG . 1 of Swarm Learning, Org - 1 through [ 0093 ] In some embodiments, Swarm Learning nodes
Org - 4 represent four separate installations of the same or works in collaboration with other Swarm Learning nodes in
related AI /ML Deep learning neural networks in four sepa the network . In some embodiments, each swarm learning
rate national regions with cross border data flow restrictions node regularly shares its deep transfer learning model fea
and disparate data privacy rules and regulations . SPIRE tures with the other nodes and incorporates their insights.
Federation represents the deep learning model features This process continues until the Swarm Learning nodes train
derived from the Federated and / or Centralized Data in each the model to desired state .
national region . The SPIRE Federation employs deep learn [ 0094 ] FIG . 2 depicts a partial view of the signal data
ing transfer to synchronize the deep learning model features signature classifier system 112 with an input signal data
across the four national regions (ORG - 1 through ORG - 4 ) : signature recording 101 captured using a physical hardware
[ 0086 ] In some embodiments, Swarm Learning may device , microphone 200 ; such that the signal data signature
include five components , connected to form a network : signal is captured as a .wav file 201 , or any other type of
[ 0087 ] Swarm Learning ( SL ) nodes — These nodes run a computer readable signal data signature signal formatted
user -defined Machine Learning algorithm . This algo file, and is then pre -processed 202. Signal Data Signature
rithm is called the Swarm Learning ML Program . The Pre -Processing 202 imposes a few , basic standards upon the
Swarm Learning ML Program program is responsible sample. This filter acts to address three quality - centric
for training and updating the model in an iterative concerns, specifically ; Stereo to Mono Compatibility, Peak
fashion . In some embodiments, the Swarm Learning Input Loudness Level , and Attenuation of Unrelated Low
ML program may be a Keras (TensorFlow 2 backend) Frequencies.
or Py Torch based Machine Learning algorithm that is [ 0095 ] In some embodiments, the first function of the filter
implemented using Python3, or any other suitable which addresses Stereo to Mono Compatibility , combines
tooling for the machine learning algorithm . the two channels of stereo information into one single mono
US 2022/0215248 A1 Jul. 7. 2022
9

representation. This ensures that only a single perspective of [ 0101 ] In some embodiments, referring to FIG . 4 , mem
the signal is being considered or analyzed at one time . bers 402-404 ( e.g. , clients ) of the exemplary computer
[ 0096 ] In some embodiments , once the signal is summed based system and platform 400 may include virtually any
to mono , it is then normalized , and brought up to its loudest computing device capable of receiving and sending a mes
possible peak level while preserving all other spectral char sage over a network (e.g. , cloud network ), such as network
acteristics of the source ; including frequency content, 405 , to and from another computing device, such as servers
dynamic range as well as the signal to noise ratio of the 406 and 407 , each other, and the like . In some embodiments,
sound . the member devices 402-404 may be personal computers ,
[ 0097] Finally, in some embodiments, the last step is to multiprocessor systems , microprocessor- based or program
remove any unwanted low frequency noises that could mable consumer electronics, network PCs , and the like . In
obscure the analysis of the target sound of the source file. some embodiments, one or more member devices within
This is achieved by implementing a High Pass Filter, with a member devices 402-404 may include computing devices
Cutoff of 80 hz at a slope of -36 dB / 8va (Oct) . that typically connect using a wireless communications
medium such as cell phones, smart phones , pagers , walkie
[ 0098 ] In some embodiments , once signal data signature talkies , radio frequency (RF ) devices , infrared ( IR) devices,
preprocessing is complete, feature extraction algorithms CBs , integrated devices combining one or more of the
operate on the pre -processed signal data signature file gen preceding devices, or virtually any mobile computing
erating feature extraction 203 which along with or without device, and the like . In some embodiments, one or more
symptoms 204 , medical history 205 are feed into a feature member devices within member devices 402-404 may be
vector 206. The feature vector 206 is used as an input to train devices that are capable of connecting using a wired or
machine learning model ( s) 113 which result in an ensemble wireless communication medium such as a PDA , POCKET
of n classifiers 207. The ensemble of n classifiers is used to PC , wearable computer , a laptop , tablet , desktop computer,
define the natural boundaries 114 in the training dataset . a netbook , a video game device, a pager, a smart phone, an
[ 0099 ] FIG . 3 depicts an illustrative signal data signature ultra -mobile personal computer (UMPC ) , and / or any other
classifier system in accordance with aspects of embodiments device that is equipped to communicate over a wired and / or
of the present disclosure. In some embodiments, referring to wireless communication medium ( e.g. , NFC , RFID , NBIOT,
FIG . 3 , the signal data signature is captured by a mobile 3G , 4G , 5G , GSM , GPRS , WiFi, WiMax, CDMA , satellite ,
phone or other mobile device using an app or a web client ZigBee , etc. ). In some embodiments, one or more member
( 301 ) . The signal data signature passes through a pre devices within member devices 402-404 may include may
processing filter as describer for (202 ) above and for ( 302 ) run one or more applications, such as Internet browsers ,
in this figure. The signal data signature is filtered using a mobile applications , voice calls , video games , videoconfer
Hidden Markov Model (HMM ) help direct signal data encing, and email , among others. In some embodiments , one
signatures (303 ) to the correct classifiers. The data then or more member devices within member devices 402-404
flows through a parallel data pipeline (304 ) . The signal data may be configured to receive and to send web pages , and the
signature is passed to a comparison classifier (305 ) for the like . In some embodiments, an exemplary specifically pro
purpose of determining whether or not the submitted signal grammed browser application of the present disclosure may
data signature matches the baseline cluster of signal data be configured to receive and display graphics, text, multi
signatures for the user . Simultaneously, the data is passed to media, and the like , employing virtually any web based
multiple identical convolutional neural network classifiers language, including, but not limited to Standard Generalized
( CNN ) ( 306 ) existing as instances in identical environments Markup Language ( SMGL ) , such as HyperText Markup
trained with randomly selected signal data signatures from a Language ( HTML ) , a wireless application protocol (WAP ),
large pool of calibration quality signal data signatures clas a Handheld Device Markup Language (HDML ) , such as
sify the incoming signal data signature (306 ) . The relative Wireless Markup Language ( WML ) , WMLScript , XML ,
probability of a signal data signature matching a signal data JavaScript, and the like . In some embodiments, member
signature library in each classifier is passed to a determin device within member devices 402-404 may be specifically
istic oracle /algorithm ( 307) provides a a diagnosis programmed by either Java, .Net , QT, C , C ++ and /or other
[ 0100 ] FIG . 4 depicts a block diagram of an exemplary suitable programming language. In some embodiments, one
computer-based system and platform 400 in accordance with or more member devices within member devices 402-404
one or more embodiments of the present disclosure. How may be specifically programmed include or execute an
ever, not all of these components may be required to practice application to perform a variety of possible tasks , such as ,
one or more embodiments, and variations in the arrangement without limitation , messaging functionality, browsing,
and type of the components may be made without departing searching, playing , streaming or displaying various forms of
from the spirit or scope of various embodiments of the content, including locally stored or uploaded messages ,
present disclosure. In some embodiments, the illustrative images and / or video , and / or games .
computing devices and the illustrative computing compo [ 0102 ] In some embodiments, the exemplary network 405
nents of the exemplary computer -based system and platform may provide network access , data transport and / or other
400 may be configured to manage a large number of services to any computing device coupled to it . In some
members and concurrent transactions, as detailed herein . In embodiments, the exemplary network 405 may include and
some embodiments, the exemplary computer -based system implement at least one specialized network architecture that
and platform 400 may be based on a scalable computer and may be based at least in part on one or more standards set
network architecture that incorporates varies strategies for by, for example, without limitation , Global System for
assessing the data, caching, searching, and / or database con Mobile communication (GSM) Association, the Internet
nection pooling . An example of the scalable architecture is Engineering Task Force ( IETF ) , and the Worldwide Interop
an architecture that is capable of operating multiple servers . erability for Microwave Access (WiMAX ) forum . In some
US 2022/0215248 A1 Jul. 7. 2022
10

embodiments, the exemplary network 405 may implement Request Broker Architecture ( CORBA) , HTTP (Hypertext
one or more of a GSM architecture , a General Packet Radio Transfer Protocol), REST (Representational State Transfer ),
Service ( GPRS ) architecture, a Universal Mobile Telecom or any combination thereof.
munications System (UMTS ) architecture, and an evolution [ 0106 ] FIG . 5 depicts a block diagram of another exem
of UMTS referred to as Long Term Evolution (LTE ). In plary computer -based system and platform 500 in accor
some embodiments, the exemplary network 405 may dance with one or more embodiments of the present disclo
include and implement, as an alternative or in conjunction sure . However, not all of these components may be required
with one or more of the above, a WiMAX architecture to practice one or more embodiments, and variations in the
defined by the WiMAX forum . In some embodiments and , arrangement and type of the components may be made
optionally, in combination of any embodiment described without departing from the spirit or scope of various
above or below, the exemplary network 405 may also embodiments of the present disclosure . In some embodi
include , for instance, at least one of a local area network ments, the member computing devices 502a , 502b thru 502n
(LAN ), a wide area network (WAN ), the Internet, a virtual shown each at least includes a computer - readable medium ,
LAN ( VLAN ), an enterprise LAN , a layer 3 virtual private such as a random - access memory (RAM ) 508 coupled to a
network (VPN) , an enterprise IP network , or any combina processor 510 or FLASH memory . In some embodiments,
tion thereof. In some embodiments and, optionally, in com the processor 510 may execute computer - executable pro
bination of any embodiment described above or below, at gram instructions stored in memory 508. In some embodi
least one computer network communication over the exem ments, the processor 510 may include a microprocessor, an
plary network 405 may be transmitted based at least in part ASIC , and / or a state machine . In some embodiments, the
on one of more communication modes such as but not processor 510 may include , or may be in communication
limited to : NFC , RFID , Narrow Band Internet of Things
with , media , for example computer - readable media, which
stores instructions that, when executed by the processor 510 ,
(NBIOT ) , ZigBee , 3G , 4G , 5G , GSM , GPRS , WiFi, WiMax , may cause the processor 510 to perform one or more steps
CDMA, satellite and any combination thereof. In some described herein . In some embodiments, examples of com
embodiments, the exemplary network 405 may also include puter - readable media may include, but are not limited to , an
mass storage , such as network attached storage (NAS ), a electronic , optical , magnetic , or other storage or transmis
storage area network (SAN ), a content delivery network sion device capable of providing a processor, such as the
( CDN) or other forms of computer or machine readable processor 510 of client 502a , with computer - readable
media . instructions. In some embodiments, other examples of suit
[ 0103 ] In some embodiments, the exemplary server 406 or able media may include, but are not limited to , a floppy disk ,
the exemplary server 407 may be aa web server (or a series CD - ROM , DVD , magnetic disk, memory chip , ROM ,
of servers ) running a network operating system , examples of RAM , an ASIC , a configured processor, all optical media , all
2

which may include but are not limited to Microsoft Win magnetic tape or other magnetic media , or any other medium
dows Server, Novell NetWare, or Linux . In some embodi from which a computer processor can read instructions .
ments, the exemplary server 406 or the exemplary server Also , various other forms of computer - readable media may
407 may be used for and /or provide cloud and / or network transmit or carry instructions to a computer, including a
computing. Although not shown in FIG . 4 , in some embodi router, private or public network, or other transmission
ments, the exemplary server 406 or the exemplary server device or channel, both wired and wireless . In some embodi
407 may have connections to external systems like email, ments, the instructions may comprise code from any com
SMS messaging , text messaging, ad content providers, etc. puter -programming language, including, for example, C ,
Any of the features of the exemplary server 406 may be also C ++ , Visual Basic , Java, Python, Perl, JavaScript, and etc.
implemented in the exemplary server 407 and vice versa . [ 0107] In some embodiments, member computing devices
[ 0104 ] In some embodiments , one or more of the exem 502a through 502n may also comprise a number of external
plary servers 406 and 407 may be specifically programmed or internal devices such as a mouse , a CD - ROM , DVD , a
to perform , in non - limiting example, as authentication serv physical or virtual keyboard , a display, or other input or
ers , search servers , email servers , social networking services output devices. In some embodiments, examples of member
servers, SMS servers, IM servers , MMS servers , exchange computing devices 502a through 502n (e.g. , clients ) may be
servers , photo -sharing services servers , advertisement pro any type of processor- based platforms that are connected to
viding servers , financial /banking - related services servers , a network 506 such as , without limitation , personal com
travel services servers, or any similarly suitable service -base puters, digital assistants, personal digital assistants, smart
servers for users of the member computing devices 401-404 . phones , pagers, digital tablets , laptop computers, Internet
appliances, and other processor -based devices. In some
[ 0105] In some embodiments and, optionally, in combi embodiments, member computing devices 502a through
nation of any embodiment described above or below , for 502n may be specifically programmed with one or more
example, one or more exemplary computing member application programs in accordance with one or more prin
devices 402-404 , the exemplary server 406 , and / or the ciples /methodologies detailed herein . In some embodiments,
exemplary server 407 may include a specifically pro member computing devices 502a through 502n may operate
grammed software module that may be configured to send, on any operating system capable of supporting a browser or
process, and receive information using a scripting language, browser - enabled application , such as MicrosoftTM Win
a remote procedure call , an email , a tweet, Short Message dowsTM , and /or Linux . In some embodiments, member com
Service ( SMS ) , Multimedia Message Service (MMS ) , puting devices 502a through 502n shown may include , for
instant messaging (IM) , internet relay chat (IRC ) , mIRC , example , personal computers executing a browser applica
Jabber, an application programming interface , Simple tion program such as Microsoft Corporation's Internet
Object Access Protocol ( SOAP ) methods, Common Object ExplorerTM , Apple Computer, Inc.'s SafariTM , Mozilla Fire
US 2022/0215248 A1 Jul. 7. 2022
11

fox , and / or Opera. In some embodiments, through the mem [0114] wherein the computer program , when
ber computing client devices 502a through 502n , users , executed, causes the processor to perform steps to :
512a through 502n , may communicate over the exemplary [ 0115 ] receive a signal data signature recording
network 506 with each other and / or with other systems from at least one data source ;
and / or devices coupled to the network 506. As shown in [ 0116 ] wherein the memory unit is configured to
FIG . 5 , exemplary server devices 504 and 513 may be also store the data sources created by the physical
coupled to the network 506. In some embodiments, one or interface on a temporary basis ;
more member computing devices 502a through 502n may be [ 0117] receive a dataset of labeled signal data
mobile clients . signature recordings including signal data signa
[ 0108 ] In some embodiments, at least one database of ture recording labels ;
exemplary databases 507 and 515 may be any type of [ 0118 ] wherein the memory unit is configured to
database , including a database managed by a database store the signal data signature recording and
management system ( DBMS ) . In some embodiments, an dataset of labeled signal data signature record
exemplary DBMS -managed database may be specifically ings created by the physical interface on a
programmed as an engine that controls organization, stor temporary basis ;
age , management, and/or retrieval of data in the respective [ 0119 ] identify, using at least one machine learning
database . In some embodiments, the exemplary DBMS model , boundaries within the dataset of labeled
managed database may be specifically programmed to pro signal data signature recordings;
vide the ability to query , backup and replicate, enforce rules, [ 0120 ] classify the signal data signature recording
provide security, compute, perform change and access log to produce an output label using a compendium of
ging , and / or automate optimization . In some embodiments, signal data signature classifiers based on the
the exemplary DBMS-managed database may be chosen boundaries within the dataset of labeled signal
from Oracle database , IBM DB2 , Adaptive Server Enter data signature recordings;
prise, FileMaker, Microsoft Access , Microsoft SQL Server, [ 0121 ] determine an output type of the signal data
MySQL , PostgreSQL , and a NoSQL implementation . In signature recording; and
some embodiments, the exemplary DBMS -managed data [ 0122 ] display the output label on a display media .
base may be specifically programmed to define each respec [ 0123 ] While one or more embodiments of the present
tive schema of each database in the exemplary DBMS , disclosure have been described, it is understood that these
according to a particular database model of the present embodiments are illustrative only, and not restrictive, and
disclosure which may include aa hierarchical model , network that many modifications may become apparent to those of
model , relational model , object model , or some other suit ordinary skill in the art , including that various embodiments
able organization that may result in one or more applicable of the inventive methodologies, the illustrative systems and
data structures that may include fields, records, files, and / or platforms, and the illustrative devices described herein can
objects. In some embodiments, the exemplary DBMS-man be utilized in any combination with each other. Further still ,
aged database may be specifically programmed to include the various steps may be carried out in any desired order
metadata about the data that is stored . (and any desired steps may be added and / or any desired
[ 0109 ] In some embodiments, the exemplary inventive steps may be eliminated ).
computer-based systems/platforms, the exemplary inventive What is claimed is :
computer-based devices , and / or the exemplary inventive 1. A signal data signature detection system , comprising:
computer-based components of the present disclosure may a physical hardware device consisting of a memory unit
be specifically configured to operate in aa cloud computing and processor;
architecture 525 such as , but not limiting to : infrastructure a wherein the memory unit is configured to store a
service ( IaaS ) 710 , platform as a service ( PaaS ) 708 , and / or
9
computer program or computer programs created by
software as a service ( SaaS ) 706 using a web browser, the physical interface on a temporary basis ;
mobile app , thin client, terminal emulator or other endpoint wherein the computer program , when executed, causes
704. FIGS . 6 and 7 illustrate schematics of exemplary the processor to perform steps to :
implementations of the cloud computing / architecture ( s) in receive a signal data signature recording from at least
which the exemplary inventive computer -based systems /
platforms, the exemplary inventive computer -based devices , one data source ;
and /or the exemplary inventive computer-based components receive a dataset of labeled signal data signature
of the present disclosure may be specifically configured to recordings;
operate . wherein the dataset of labeled signal data signa
[ 0110 ] The aforementioned examples are, of course , illus ture recordings comprises a dataset of signal
trative and not restrictive . data signature recording labels ;
[ 0111 ] At least some aspects of the present disclosure will wherein each labeled signal data signature record
now be described with reference to the following numbered ing of the dataset of labeled signal data signa
clauses . ture recordings is associated with at least one
1. A signal data signature detection system , comprising: signal data signature recording label of the
dataset of signal data signature recording labels:
[ 0112 ] a physical hardware device comprising of a identify, using at least one machine learning model ,
memory unit and processor; boundaries within the dataset of labeled signal
[ 0113 ] wherein the memory unit is configured to store data signature recordings;
a computer program or computer programs created classify the signal data signature recording to pro
by the physical interface on a temporary basis ; duce an output label using the compendium of
US 2022/0215248 A1 Jul. 7. 2022
12

signal data signature classifiers based on the at instructing, by the at least one processor, a display media
least one transfer learning framework; to display the output type.
determine an output type of the signal data signature 9. The method as recited in claim 8 , wherein the at least
recording based at least in part on the output label; one transfer learning framework comprises a swarm learning
and framework .
display the output type on a display media . 10. The method as recited in claim 9 , wherein each signal
2. The signal data signature detection system as recited in data signature classifier in the compendium of signal data
claim 1 , wherein the at least one transfer learning framework signature classifiers is a swarm node in the swarm learning
comprises a swarm learning framework . framework .
3. The signal data signature detection system as recited in 11. The method as recited in claim 8 , further comprising :
claim 2 , wherein each signal data signature classifier in the determining, by the at least one processor, an inductive
compendium of signal data signature classifiers is a swarm bias in the dataset of labeled signal data signature
node in the swarm learning framework . recordings based at least in part on the dataset of signal
4. The signal data signature detection system as recited in data signature recording labels ; and
claim 1 , wherein the computer program , when executed , utilizing, by the at least one processor, the at least one
causes the processor to perform steps to : transfer learning framework comprising inductive
determine an inductive bias in the dataset of labeled signal transfer to apply the inductive bias to the compendium
data signature recordings based at least in part on the of signal data signature classifiers.
dataset of signal data signature recording labels ; and 12. The method as recited in claim 8 , wherein at least one
utilize the at least one transfer learning framework com signal data signature classifiers in the compendium of signal
prising inductive transfer to apply the inductive bias to data signature classifiers is a deep learning neural network .
the compendium of signal data signature classifiers . 13. The method as recited in claim 8 , further comprising:
5. The signal data signature detection system as recited in receiving , by the at least one processor, a target audio
claim 1 , wherein at least one signal data signature classifiers recording distribution associated with the output type ;
in the compendium of signal data signature classifiers is a wherein the output label comprises at least one of a
deep learning neural network . target start time or a target end time of the signal data
6. The signal data signature detection system as recited in signature recording; and
claim 1 , wherein the computer program , when executed , modifying, by the at least one processor, the signal data
causes the processor to perform steps to : signature recording to produce a modified signal data
receive a audio recording distribution associated signature recording based at least in part on the outpu
with the output type ; label .
wherein the output label comprises at least one of a 14. The method as recited in claim 8 , wherein the com
target start time or a target end time of the signal data pendium of signal data signature classifiers are trained based
signature recording ; and on a balanced training dataset.
modify the signal data signature recording to produce a 15. A non -transitory computer readable medium having
modified signal data signature recording based at least software instructions stored thereon , the software instruc
in part on the output label. tions configured to cause at least one processor to perform
7. The signal data signature detection system as recited in steps comprising:
claim 1 , wherein the compendium of signal data signature receiving a signal data signature recording from at least
classifiers are trained based on a balanced training dataset. one data source ;
8. A signal data signature detection method, comprising : receiving a dataset of labeled signal data signature record
receiving, by at least one processor, a signal data signature ings ;
recording from at least one data source ; wherein the dataset of labeled signal data signature
receiving, by the at least one processor, a dataset of recordings comprises a dataset of signal data signa
labeled signal data signature recordings; ture recording labels ;
wherein the dataset of labeled signal data signature wherein each labeled signal data signature recording of
recordings comprises a dataset of signal data signa the dataset of labeled signal data signature record
ture recording labels ; ings is associated with at least one signal data
wherein each labeled signal data signature recording of signature recording label of the dataset of signal data
the dataset of labeled signal data signature record signature recording labels:
ings is associated with at least one signal data identifying, using at least one machine learning model ,
signature recording label of the dataset of signal data boundaries within the dataset of labeled signal data
signature recording labels : signature recordings;
identify, by the at least one processor, using at least one classifying the signal data signature recording to produce
machine learning model, boundaries within the dataset an output label using the compendium of signal data
of labeled signal data signature recordings; signature classifiers based on the at least one transfer
classifying, by the at least one processor, the signal data learning framework ;
signature recording to produce an output label using the determining an output type of the signal data signature
compendium of signal data signature classifiers based recording based at least in part on the output label; and
on the at least one transfer learning framework ; instructing a display media to display the output type .
determining, by the at least one processor, an output type 16. The non - transitory computer readable medium as
of the signal data signature recording based at least in recited in claim 15 , wherein the at least one transfer learning
part on the output label ; and framework comprises a swarm learning framework .
US 2022/0215248 A1 Jul . 7 , 2022
13

17. The non - transitory computer readable medium as

recited in claim 16 , wherein each signal data signature
classifier in the compendium of signal data signature clas
sifiers is a swarm node in the swarm learning framework .
18. The non -transitory computer readable medium as
recited in claim 15 , wherein the software instructions are
further configured to cause the at least one processor to
perform steps comprising:
determine an inductive bias in the dataset of labeled signal
data signature recordings based at least in part on the
dataset of signal data signature recording labels ; and
utilize the at least one transfer learning framework com
prising inductive transfer to apply the inductive bias to
the compendium of signal data signature classifiers.
19. The non - transitory computer readable medium as
recited in claim 15 , wherein at least one signal data signature
classifiersa in the compendium of signal data signature clas
sifiers is a deep learning neural network .
20. The non - transitory computer readable medium as
recited in claim 15 , wherein the software instructions are
further configured to cause the at least one processor to
perform steps comprising:
receiving a target audio recording distribution associated
with the output type;
wherein the output label comprises at least one of a
target start time or a target end time of the signal data
signature recording; and
modifying the signal data signature recording to produce
a modified signal data signature recording based at least
in part on the output label .
*