11th

11th IFAC
IFAC Symposium
Symposium on on Robot
Robot Control
Control
11th
11th IFAC
August
IFAC Symposium
26-28, on
on Robot
2015. Salvador,
Symposium Salvador,
Robot Control
BA, Brazil
Control
August
11th 26-28,
IFAC 2015.
Symposium on RobotBA, Brazil
Control
August
August 26-28, 2015. Salvador, BA, Brazil
26-28, 2015. Salvador, BA, Available online at www.sciencedirect.com
Brazil
August 26-28, 2015. Salvador, BA, Brazil

ScienceDirect
IFAC-PapersOnLine 48-19 (2015) 154–159
On
On the
the Brain
Brain Computer
Computer Robot
Robot Interface
Interface (BCRI)
(BCRI) to
to
On the Brain Computer
On the Brain Computer Robot
Robot Interface
Interface (BCRI)
(BCRI) to
to
Control Robots
Control Robots
Control Robots
Control Robots
U.
U. Sanchez-Fraire
Sanchez-Fraire
∗∗ V. Parra-Vega ∗∗ D. Martinez-Peon ∗∗
∗∗ V. Parra-Vega ∗∗ D. Martinez-Peon ∗∗
G. Sep
SepU. U.
U. Sanchez-Fraire
Sanchez-Fraire
úlveda-Cervantes ∗∗∗∗ V.
V. Parra-Vega
Parra-Vega
A. Sanchez-Orta ∗ D.
D. ∗∗Martinez-Peon
Martinez-Peon
A.J. Mu Muñoz-Vázquez
∗∗
∗∗
ñoz-Vázquez ∗∗
G.
G. Sep úlveda-Cervantes
Sanchez-Fraire
Sepúlveda-Cervantes
úlveda-Cervantes ∗∗∗ A.
∗∗∗
∗∗∗ V. A. Sanchez-Orta
Parra-Vega
A. Sanchez-Orta D.
Sanchez-Orta ∗ A.J. ∗ A.J.
Martinez-Peon
A.J. Mu Muñoz-Vázquez
∗∗
ñoz-Vázquez ∗∗∗
G.
G. Sepúlveda-Cervantes
∗∗∗ ∗
A. Sanchez-Orta A.J. Muñoz-Vázquez
∗∗ Robotics and Advanced Manufacturing, Cinvestav, Saltillo Campus, Mexico,
∗ Robotics
∗ Robotics and Advanced
and Advanced Manufacturing,
Manufacturing, Cinvestav,
Cinvestav, Saltillo
Saltillo Campus,
Campus, Mexico,
Mexico,
∗ Robotics
Robotics and and Advanced
Advanced Manufacturing,
[e-mail:
[e-mail: Cinvestav,
[email protected],
[email protected],
Manufacturing, Cinvestav, Saltillo
Saltillo Campus,
Campus, Mexico,
Mexico,
[e-mail:
[e-mail: [email protected],
[email protected],
(vparra,anand.sanchez,amunozv)@cinvestav.mx].
(vparra,anand.sanchez,amunozv)@cinvestav.mx].
[e-mail: [email protected],
(vparra,anand.sanchez,amunozv)@cinvestav.mx].
∗∗ Biomedical
∗∗ (vparra,anand.sanchez,amunozv)@cinvestav.mx].
Physics and and Engineering,
Engineering, Cinvestav,
Cinvestav, Monterrey Monterrey Campus, Campus,
∗∗ Biomedical Physics
(vparra,anand.sanchez,amunozv)@cinvestav.mx].
∗∗ Biomedical Physics and Engineering, Cinvestav, Monterrey Campus,
∗∗ Biomedical Physics and Engineering, Cinvestav, Monterrey Campus,
Biomedical Mexico.,
Mexico.,
Physics (e-mail:
(e-mail:
and [email protected])
[email protected])
Engineering, Cinvestav, Monterrey Campus,
∗∗∗ Mexico.,
Mexico.,
∗∗∗ Cidetec-IPN, (e-mail:
(e-mail:
Mexico, [email protected])
[email protected])
(e-mail: [email protected])
∗∗∗ Cidetec-IPN,
Mexico.,
∗∗∗ Cidetec-IPN, Mexico,
(e-mail: (e-mail: [email protected])
[email protected])
Mexico, (e-mail: [email protected])
∗∗∗ Cidetec-IPN,
Cidetec-IPN, Mexico, Mexico, (e-mail: (e-mail: [email protected])
[email protected])
Abstract:
Abstract: Brain Brain Computer
Brain Computer Interfaces
Computer Interfaces
Interfaces (BCI) (BCI)
(BCI) have have emerged
have emerged
emerged as as
as aaa solid,
solid, though
solid, though promising
though promising
promising yet yet incipient,
yet incipient,
incipient,
Abstract:
Abstract:
mechanismBrain Computer
to communicate
communicate Interfaces
simple and and (BCI)
abstract have emerged
high-level as aa solid, though
electroencephalography promising(EEG) yet incipient,
commands
mechanism
Abstract:
mechanism to
Brain
to Computer
communicate simple
Interfaces
simple and abstract
(BCI)
abstract have high-level
emerged
high-level electroencephalography
as solid, though
electroencephalography promising(EEG)
(EEG) yet commands
incipient,
commands
mechanism
by decodingto
decoding
by decoding
mechanism communicate
brain
brain
to rhythms.
rhythms. When
communicate simple
When
When
simple BCI
BCI and
and areabstract
are used
used for
abstract forhigh-level
for online
online commands
high-level electroencephalography
commands
commands of
of aaa kinematic
electroencephalography kinematic
kinematic or or(EEG)
or dynamic
dynamic
(EEG) commands
external
external
commands
by
by decoding
agent, either brain
brain
real rhythms.
rhythms.
or virtual When
ones, BCI
BCI
human are
are used
used
perceives for online
online
the task commands
with of
of
different a kinematic
attributes, or dynamic
dynamic
thus different external
external
brain
agent,
by
agent, either
decoding
either real
brain
real or
or virtual
rhythms.
virtual ones,
When
ones, human
BCI
human areperceives
used
perceives for the task
online
the task with
commands
with different
of
different a attributes,
kinematic
attributes, thus
or different
dynamic
thus different brain
external
brain
agent,
regions
regions
agent, either
are
are
either real or
activated
activated
real or virtual
depending
depending
virtual ones,
ones, human
on
on the
the
human perceives
characteristic
characteristic
perceives the
the task
of
of
task the
the with
with different
perceptual
perceptual
different attributes,
task
task and
and
attributes, the
thethus different
cognitive
cognitive
thus different brain
process
process
brain
regions
regions
involved.areareHence,
activated
activated depending
theredepending
arises the theon on the characteristic
the
question characteristic
on how ofperceived
theof
how the the perceptual
the perceptualkinematic task or
task and
and the cognitive
the
dynamic cognitive process
process
task attributes
attributes
involved.
regions
involved. Hence,
areHence, there
activated
there arises
depending
arises theon question
question on
the characteristic
on perceived
theofperceived
how the the perceptual
perceived kinematic
kinematic task or or
and dynamic
the cognitive
dynamic task
task process
attributes
involved.
affects
affects
involved. the
the Hence,
ability
ability
Hence, there
of
of human
human
there arises
arises to
to the
produce
produce
the question
questiona
a BCI
BCI on
on how
command.
command.
how the In
In this
this
perceived paper,
paper, kinematic
a
a
kinematicBrain
Brain or dynamic
Computer
Computer
or dynamic task
Robot
Robot
task attributes
Interface
Interface
attributes
affects
affects the ability of human tocontains
produce aa BCI command. Ina this paper, aa Brain Computer Robot Interface
(BCRI)the
(BCRI)
affects
(BCRI) is ability
is
the
is
proposed,
proposed,
ability
proposed,
of
of human
whichto
which
human
which
produce
contains
tocontains
produceas as
asa BCI
aa core
BCI
a
command.
core elementIn
element
command.
core element Inaa this
BCIpaper,
BCI
this
BCI
to
to decode
paper,
to a Brain
decode
decode Brain Computer
commands
commands
Computerfor
commands
Robot
for
for
Robot a
Interface
aa dynamical
dynamical
Interface
dynamical
(BCRI)
external
external
(BCRI) is proposed,
agent,
agent,
is such
such
proposed, as
aswhich
a
a
which robot.
robot. contains
To
To
contains as
convey
convey
as a
a core
the
the
core element
attributes
attributes
element of
of a
a a
aBCI
dynamic
dynamic
BCI to
to decode
perceptual
perceptual
decode commands
commands task,
task, for
advanced
advanced
for a
a dynamical
tools
tools of
dynamical of
of
external
external agent,
dynamicsagent, such as
such
and control
control asofaa robots
robot. are
robot. To convey
To convey
borrowed thetoattributes
the attributes
integrateofof
theaaa BCRI,
dynamic
dynamic
BCRI, perceptual
perceptual
complying task,
task,
with advanced
realadvanced
time constraints tools
tools
constraints of
dynamics
external
dynamics and
agent,
and such
control as of
ofa robots
robot.
robots are
To
are borrowed
convey
borrowed the to
to integrate
attributes
integrate the
of
the dynamic
BCRI, complying
perceptual
complying with real
task,
with real time
advanced
time tools
constraints of
dynamics
and
and behaviors. and
and control
behaviors.
behaviors.
dynamics of
of robots
Experimental
Experimental
control robots are
are borrowed
results
results shows
shows that
borrowed to
to integrate
that
that the
the abilitythe
ability
ability
integrate thetotoBCRI,
to generate
generate
BCRI, complying
BCI
BCI commands
complying with
with real
commands
commands istime
is
realis dueconstraints
due
time to
to aaa lower
lower
lower
constraints
and
and
user behaviors.
frustration, Experimental
Experimental
which is results
results
obtained shows
shows
when that
the the
the ability
perceptual taskto generate
generate
matches BCI
BCI
the commands
dynamic is due
due
attributes to
to a lower
exhibited
user
and
user frustration,
behaviors.
frustration, which
Experimental
which is
is obtained
results
obtained when
shows
when the
thatperceptual
the abilitytaskto matches
generate the
BCI dynamic
commands attributes
is due exhibited
to a lower
user
by
user frustration,
by aa dynamic
dynamic
frustration, which
robot.
robot.
whichIn is
is obtained
In contrast,
contrast,
obtained when
whenwhen
when the the
the the
robot
robot
the
perceptual
perceptual task
task
behaves with
behaves
perceptual with matches
kinematicthe
task matches
kinematic
matches the
the
dynamic
dynamic
attributes
attributes
dynamiconly,
attributes
attributes
only,
only,
exhibited
exhibited
high frustration
high
attributes frustration
exhibited
by
by a
a
impedesdynamic
dynamic robot.
robot.
the generation
generation In
In contrast,
contrast,
of BCI
BCI commands.when
when
commands. the
the robot
robot behaves
behaves with
with kinematic
kinematic attributes
attributes only, high
high frustration
frustration
impedes
by a
impedesdynamicthe
the robot.
generation In of
contrast,
of BCI when
commands. the robot behaves with kinematic attributes only, high frustration
impedes
impedes the
the generation
generation of
of BCI
BCI commands.
commands.
© 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Keywords: Brain-Computer
Keywords: Brain-Computer Interface, Interface, HapticHaptic Interaction,
Interaction, Dynamic
Dynamic Robots, Robots, Mobile Mobile Robots
Robots
Keywords:
Keywords: Brain-Computer
Brain-Computer Interface, Interface, HapticHaptic Interaction,
Interaction, Dynamic
Dynamic Robots, Robots, Mobile Mobile Robots
Robots
Keywords: Brain-Computer Interface, Haptic Interaction, Dynamic Robots, Mobile Robots
1. INTRODUCTION
1. INTRODUCTION that have
that have been been largely
largely overlooked,
overlooked, in in particular
particular by by thethe robot
robot
1.
1. INTRODUCTION
INTRODUCTION that
that
controlhave
have been
been
research largely
largely
community. overlooked,
overlooked, in
in particular
particular by
by the
the robot
robot
1. INTRODUCTION control
that
controlhave research community.
been largely overlooked, in particular by the robot
control research
control research community.
research community.
community.
Over four
Over four decades
decades ago, ago, the the foundations
foundations of of BCIBCI werewere estab- estab- ThereThere are are strongstrong limitations
limitations to to carry
carry out out aa statistically
statistically valid valid
Over
Over four
lished four thedecades
in the seminalago,
decades ago, the
the foundations
foundationspaper
and comprehensive
comprehensive of
of BCI
paper BCI were
were(1973),
of Vidal
Vidal estab-
estab- There
(1973), There
studies are
arewith strong
strong
BCI, limitations
limitations
because to
to carry
carry
EEG-based out
out aa statistically
commandsstatistically are valid
valid
diffi-
lished
Over
lished in
four seminal
decades and
ago, the foundations of of
BCI were estab- studies
There are with BCI, because
strong limitations EEG-based
to carry outcommands commands
a statistically are diffi-
valid
lished in
which
which
lished in the
the seminal
covered
covered
in the seminal
the whole
the
seminal
and
and comprehensive
whole
and comprehensive
spectrum of
spectrum
comprehensive of what
what paper
paper
paperis
of
of Vidal
of Vidal (1973),
is nowadays
nowadays
Vidal known studies
(1973),
known
(1973), studies
cult
cult
studies
with
with BCI,
to generate
to generate
with BCI,
BCI, for because
for because
all usersunder
all
because
EEG-based
EEG-based
usersunder
EEG-based commandsconditions
same controlled
same controlled
commands
are
are diffi-
diffi-
conditions
are diffi-
which
which
as a covered
covered
BCI, the
the
essentially whole
whole a spectrum
spectrum
system to of
of what
what
communicate is
is nowadays
nowadays
humans known
knownand cult
cult
over to
totimegenerate
generate
with an for
for all
all
efficient usersunder
usersunder
mental same
same
command controlled
controlled(Kosslyn, conditions
conditions
2001).
as aa BCI,
which
as BCI,
coveredessentially
the whole
essentially a system
system
spectrum to ofcommunicate
what is nowadays humans knownand cult over to time with an
generate an forefficient mental command
all usersunder command
same controlled (Kosslyn, 2001).
conditions
as aa BCI,
BCI, using
machines
machines
as using brain aaaEEG
essentially
brain
essentially EEGsystem
system
to
to communicate
waves,
waves,
to communicate
(Wolpaw, 2002).
(Wolpaw,
communicate
humans
humans
2002).
humans Though
Though
and
and over
and over time
However,
However,
over time the
time
with
withmain
the
with an efficient
main
an concern mental
efficient
concern
efficient mental
has been
has
mentalbeen command
mainly the
mainly
command the(Kosslyn,
(Kosslyn, 2001).
2001).
understanding
understanding
(Kosslyn, 2001).
machines
machines
earlier using
using
efforts brain
brain EEG
to decipher
decipher EEG andwaves,
decode(Wolpaw,
waves, (Wolpaw,
EEG signals
signals2002).
2002). Though
Though However,
to generate
generate However,
of the the
the
complexitiesmain
main concern
concern
to decodehas
has been
been
brain mainly
mainly
signals the
theinto understanding
understanding
commands,
earlier
machines
earlier efforts
using to brain EEG and decode
waves, EEG
(Wolpaw, 2002). to Though of the
However, complexities
the main concern to decode brain
has been signals
mainly into commands,
the understanding
earlier efforts
commands
commands
earlier efforts
effortsareto
are to
decipher
towell
decipher
well and
and decode
documented,
documented,
decipher and decode
decode the EEG
the EEG
question
question
EEG
signals
signals
signals on to
on to generate
generate
whether
whether
to generate it of
it of the
the than
rather
rather
of the
complexities
complexities
than controlto
to control
to
complexities to
decode
toexternal
external
decode
brain
decodeagent,
brainlet
agent,
brain
signals
let alone into
signals
alone
signals into commands,
commands,
the dynamics
the
into dynamics
commands, or
or
commands
commands
is possible are
are
to well
well
command documented,
documented,
effectively the
the question
question
external on
on
agents whether
whether
remains it
it
an rather
rather
controllingthan
than to
to control
control
issues of external
external
the agent. agent,
agent,
Moreover, let
let alone
alone the
the
although dynamics
dynamics
it is or
or
little
is possible
is possible to
commands to command
arecommand effectivelythe
well documented,
effectively external
question
external agents
agents remains
on remains
whether an an
it rather
controllingthan to issues
control of thethe agent.agent,
external Moreover, let alone although it is
the dynamics is little
little
or
is possible
open issue, to command
(Felzer, 2001). effectively
Recently, external
several agents
research remains
fields an
set controlling
controlling
known the issues
issues
mechanisms of
of the agent.
agent.
that Moreover,
Moreover,
hinder some although
although
users to it
it is
operate little
open
is
open issue,
possible
issue, (Felzer,
to command
(Felzer, 2001).
2001). Recently,
effectively externalseveral research
agents fields
remains set
an controlling
known the the issuesmechanisms of the that that
agent.hinder
Moreover, some althoughusers to it is littleaaa
to operate
operate
openbasics
the
the
open issue,and
basics
issue, (Felzer,
and 2001). Recently,
fundamentals
fundamentals
(Felzer, 2001). Recently,
of BCI,
of BCI, in
Recently,
several
several
in a
several
research
a particular,
particular,
research
fields
fields set
researchinterpreta-
interpreta-
fields set known
set knownclearly
BCI,
BCI,
known the mechanisms
clearly
the mechanisms
workload is
workload
mechanisms that
isthat hinder
hinderas
pointed
pointed
hinder assome
some
a main
a
some main users
users
users to operate
factor.
factor.
to operate
Then, it
Then, itaa
the
the
tion basics
basics
of and
and
commands fundamentals
fundamentals
based on of
of
EEG BCI,
BCI,
to in
in aa particular,
control particular,
of external interpreta-
interpreta-
agents, BCI,
BCI,
is fair clearly
clearly
to workload
workload
acknowledge is
is
that pointed
pointed
one main as
as aissue
a main
main is factor.
factor.
the design Then,
Then, of it
itaa
tionbasics
the
tion of commands
of commands based on
and fundamentals
based on EEGEEG
of BCI,to control
to control of external
in a particular,
of external agents, BCI,
interpreta-
agents, is fair
fairclearly
to acknowledge
acknowledge
workload that that one main
is pointed main issue
as issue
a main is factor.
the design
design Then, of it
tion of commands based on EEG to control of external agents, is
is fair to
to acknowledge that one
one main issue is
is the
the design of a
(Wolpaw,
(Wolpaw,
tion 2000),
2000), even
of commands
(Wolpaw, 2000),
even
even at
aton
basedat high
high
highEEGrate
rate
rate
and
and
to and
control performance,
performance,
of external(Cheng,
performance,
(Cheng,
(Cheng,
agents, is BCI
BCI
BCI
that
that
fairthat ables
ables
to ables a user
a user
acknowledge
a user to to command
tothat
command
one main
command
a robot
a robot
a robot
issueat at a lower
atisaa lower
lower
the design of aa
of
cognitive
cognitive
cognitive
(Wolpaw,
2002, Felzer,
2002,
(Wolpaw,Felzer,2000),
2001).
2001).
2000), even
even The at high
Theatparadigm
paradigm rate
high rateof ofand
ofand performance,
learning
learning how to
how
performance, to operate(Cheng,
operate
(Cheng,aa BCI BCI that
that ables
workload.
workload. ables aa user
Fortunately
Fortunatelyuser to toenough,
command
enough,
command aa robot
robot at
the availability
the availability at aaof lower
oflower cognitive
commercially
commercially
cognitive
2002,
BCI isFelzer,
2002,is full 2001).
Felzer,
in full swing, The
2001). The paradigm
paradigm
(Curran, 2003), ofyetlearning
learning
yet deep and how to
to operate
operate aaa workload.
howfundamental
and fundamental workload.EEG
low-cost EEGFortunately
devicesenough,
Fortunately enough,
and BCIBCI the availability
thesystems
availability
systems that of of commercially
commercially
provide several
BCI
2002,
BCI in
Felzer, swing,
2001). (Curran,
The 2003),
paradigm of deep
learning how to operate low-cost
workload. devices
Fortunately and
enough, the availability that provide
of several
commercially
BCI isis in
research
research
BCI is in
full
inhas
full
swing,
swing, (Curran,
fullfocused
has focused
swing, (Curran,
on the
on
(Curran,
2003),
2003), yet
the mechanism
mechanism
2003), yet of
yet
deep
deep
of
deepvisual
visual
and
and
fundamental
andimagery
fundamental
imagery
fundamental (VI) low-cost
(VI) low-cost EEG
cognitive
cognitive
low-cost states,devices
EEG
states,
EEG devices
including
including
devices
and
and
BCI
BCI systems
systems that
andstate-of-the-art
state-of-the-art
BCI systems that
signal
signal
that
provide
provide
processing
processing
provide
several
several
several of
of
research
research
and motor has
has focused
focused
imagery on
on
(MI) the
the mechanism
mechanism
that constitute of
of visual
visual
the imagery
basicimagery
paradigms (VI)
(VI) cognitive
cognitive
various brainstates,
states, including
including
rhythms, have state-of-the-art
state-of-the-art
made possible signal
signal
the processing
processing
study of BCI of
of
by
and motor
and motor
research imagery
hasimagery
focused (MI)(MI) that
on thethat constituteofthe
mechanism
constitute the
visual basic
basic paradigms
imagery
paradigms (VI) cognitive
various brain brain rhythms,
states, including havestate-of-the-art
made possible
possible signal the study
study of BCI
processingBCI by by
of
and
used motor
in BCI imagery
to convey (MI) that
commands, constitute
(Decety, the
1996),basic paradigms
though other various
various brain
non-specialists rhythms,
rhythms,
so as to have
have
it made
made
seems possible
plausible the
the
now study
to of
of
address BCI by
the
used
and
used in
motorBCI to convey
imagery commands,
(MI) that constitute (Decety, 1996), though
the basicthough paradigms other non-specialists
various brain rhythms,so as as to tohave
it seems
seems plausible
madeplausible
possible the nowstudy to address
address
of BCIthe the
by
used in
in BCI
paradigms
paradigms
used in BCIfor
BCI
to
to convey
for
to trainingcommands,
convey
training
convey commands,
has been
has
commands,
(Decety,
(Decety, 1996),
been proposed
proposed
(Decety, 1996),
including
including
1996), though
though
other
other non-specialists
aa multi-
multi-
multi-
other non-specialists
problem
problem
non-specialistsof BCI-based
of
so
so as
BCI-based
so as to it
tocontrol
it seems
seems
control
it plausible robots,
of dynamic
of dynamic
plausible
now
now to
robots,
now towe
to address
we draw the
draw
address the
paradigms
paradigms for for
for training
training has been
has been proposed
been2009),
proposed including
including a one, problem
problem of to BCI-based control of
of dynamic robots,
robots, we we drawdraw the
disciplinary
disciplinary
paradigms
disciplinary
one,
one,
one,
(Ron-Angevin,
(Ron-Angevin,
training has
(Ron-Angevin,
2009),
proposed
2009),
and including
and
and
a multimodal
multimodal
aaa multimodal aa multi-
multi-
one,
one,
attention
attention
problem of
to
of BCI-based
robot control
robot
BCI-based control control
specialist
specialist
control of dynamic
on this
on
dynamic this issue.
issue.
robots, we draw the
the
disciplinary
(Maye, 2011), one, or (Ron-Angevin,
based on 2009),
kinesthetic and
stimuli multimodal
(Brown, one,
2005, attention
attention to
to robot
robot control
control specialist
specialist on
on this
this issue.
issue.
(Maye, 2011),
(Maye, 2011),
disciplinary one, or(Ron-Angevin,
based on on kinesthetic
kinesthetic stimuli
2009), stimuli (Brown, 2005,
and a multimodal 2005,
one, attention studytoon
The study onrobot
howcontroland under specialist
under whatonconditions
this issue.an EEG-based
conditions
(Maye, 2011),
Dechent,
Dechent,
(Maye, 2004,or
2011),
2004, or
based
orNeuper,
based 2005,
Neuper,
based on kinesthetic
2005,
on kinesthetic
Stinear, 2006).
Stinear, stimuli When
2006).
stimuli
(Brown,
(Brown,
When
(Brown, aa BCI 2005,
BCI
2005, is The
is The study on how
how and
and under what
what conditions an EEG-based
an EEG-based
Dechent, 2004, Neuper, 2005, Stinear, 2006). When aa BCI is The
mental
mental study state
state on how
can
can and
yield
yield aa under
useful
useful what
command
command conditions to control
to controlan EEG-based
EEG-based
a external
external
Dechent,
used
used toto
Dechent,
used
2004,
command
to command
command Neuper,
2004, Neuper, real
real
or
real or 2005,
virtual
or 2005,
virtual
Stinear,
external
virtualStinear,
external
2006).
external2006). agents
agentsWhen
agents
When
(Bogue,
(Bogue,
(Bogue, a BCI BCI
2010,
2010,
2010,
is The
is mental
mental
agent
study state
state
has
on
been
how
can
can a
and
yield
yield
major
under
aa useful
useful
subject
what
command
command
of
conditions to
to
multidisciplinary control
control
an aaaresearch.
external
external
used to
Brunne,
Brunne,
used to command
command
2011, McFarlan,
2011, real or
McFarlan,
real or virtual
virtual external
2008, Patil,
2008, Patil,
external 2008),
2008),agents
agents (Bogue,
simple
simple
(Bogue, regulators
regulators2010, mental
2010, agent has
agent has
state been
beencanaayield
majora useful
major subjectcommand
subject of multidisciplinary
of multidisciplinary
to control aresearch. research.
external
Brunne, 2011, McFarlan, 2008, Patil, 2008), simple regulators agent
The main
The main
mainhas been
interest
interest a major
is the
is the
the usersubject
user
user of
command
command multidisciplinary
generation
generation that that
that research.
can be
can be
Brunne,
have
have
Brunne, been
been2011, McFarlan,
proposed
proposed
2011, McFarlan, to 2008, Patil,
to control
control
2008, Patil,
aa robot,
robot,
robot,2008),
2008), simple regulators
or computed
or computed
simple torque agent
regulators
torque The has interest
been a major is subject of multidisciplinary research.
have
have been
been proposed
proposed to
to control
control aa robot, or
or computed
computed torque
torque The main
encoded
encoded
The mainintointo interest
desired
desired
interest is the
is the user command
references
references
user command
to
to
command the
the
generation
generation
controller
controller
generation of
of the
the
that
can
can be
thatexternal
external
can be
be
schemes,
schemes,
have been (Millan,
(Millan,
proposed 2004).
2004).
to A byproduct
A byproduct
control a robot,of these
of these
or studies suggests
studies
computed suggests
torque encoded into desired references to the controller of the external
schemes, (Millan, 2004). A
A byproduct of
of these encoded
agent, into
regardless desired of references
the nature to
of the
this controller
external of the
agent, external
either aaa
either
schemes,
the
the need to
need
schemes, to(Millan,
consider
consider
(Millan, 2004).
dynamical
dynamical
2004). A byproduct
properties
properties
byproduct of these
of the
of
these thestudies
plant, suggests
studies
plant,
studies suggests
an issue
an
suggests agent, regardless
issue encoded
agent, regardless
into desired of the
of the nature of
references
nature of
to thethiscontroller
this external agent,
external agent,
of the external
either
the need
the need
need to to consider
to consider dynamical
consider dynamical properties
dynamical properties
properties of of
of thethe plant,
the plant,
plant, an an issue agent, regardless of the nature of
issue agent, regardless of the nature of this external agent, either
an issue this external agent, either aa
the
Copyright
2405-8963 ©
Copyright 2015 IFAC
© 2015,
2015 IFAC 156 Hosting by Elsevier Ltd. All rights reserved.
IFAC (International Federation of Automatic Control)
156
Copyright
Peer review©
Copyright 2015
2015 IFAC
©under IFAC 156
156Control.
Copyright © 2015 responsibility
IFAC of International Federation of Automatic
156
10.1016/j.ifacol.2015.12.026
 IFAC SYROCO 2015
August 26-28, 2015. Salvador, BA, Brazil U. Sanchez-Fraire et al. / IFAC-PapersOnLine 48-19 (2015) 154–159 155

pointer in a screen, a speech synthesizer, (Bakardjian, 2010, 2. THE BCRI PROPOSAL

Birbaumer, 1999) or a real robot, (Millan, 2004, Nicolelis,
2002). Aiming at exposing the fact that commanding a robot 2.1 Components of the BCRI
with a BCI indeed stands for an extreme case of perceptual task
(the quality of the task depends on how user perceives the in- A typical BCI is depicted in Fig. 1.a where the systems have
trinsic quantitative behaviors exhibited by the controlled robot, 5 blocks, the block of EEG signals acquisition and processing
rather than by the quality of the robot control itself ) (Wolpert, where EEG commands are decodified; the FSM where the
1995, Xiao, 2006), then we hypothesize that controlling a kine- commands are translated to robot language; the robot which
matic robot is perceptually different from controlling a dynamic perform the desired task; the feedback to the user that is usually
robot, thus task perception is strikingly different depending visual. Our proposal, depicted in Fig. 1.b., where robotic tools
on the robot type so as to it affects the workload that user
experience, which may influence and limit the user ability for
BCI command. In any case, it is well-known clear that a more
effective EEG command is produced when the user is subject to
a low workload, which implies low frustration and less stress,
among other cognitive states, (Lupien, 2007). Since we are
interested of commanding a robot, it is clear that the perception
of how the robot performs affects the decision making cognitive
process. Thus, the voluntary production of user mental states
requires low user workload to react properly to command in
closed loop the robot. We conjecture that low workload can be
obtained when the controlled external agent behaves as the user
expectations under user commands. If so, this will sustain that
low workload appears in virtue of the user perception matches
the expected robot dynamic behavior to users internal mental
model representation, without conflicting to reality. In this way,
when the user is subject to low workload, user attention and
concentration grant the ability to yield brain rhythms voluntar-
ily, but a task perception that deviates from the user internal
representation may increase it. Hence, it is required a realistic
robot behavior to accomplish a meaningful brain-robot inter-
action that amounts for a robot dynamics subject to physical
laws. Though it is clear that interaction of the user and the
EEG device is not physical, user notices the causal nexus of
her/his commands when the robot moves, which increases how
critical is the timely behavior of the robot, so it is compulsory
each element that the whole system carries out accordingly,
within a common time baseline. To this end, some advanced
robotics tools are borrowed such as Euler-Lagrange dynamic
robot model, Lyapunov stability, finite state machine subject Fig. 1. a) A typical BCI is composed of an EEG-based device with its
to spatio-temporal constraints, high order sliding modes, finite corresponding signal processing to decode commands, which are sent
time stability, and dynamic simulations. In this way, the robot to a Finite State Machine (FSM) to determine desired states to the robot
fulfills the premises of a typical behavior of a real system controller, which in most cases is modeled as a simple kinematic chain.
governed by law of physics in such a way that low workload b) The proposed BCRI includes the full nonlinear dynamics in closed-
is attained by the user to produce willingly and voluntarily loop with an advanced controller that tracks trajectories provided by the
EEG events in virtue of there is a match with mental models FSM to enforce the desired dynamic behavior of the robot, in compliance
under VI, MI and kinesthetic imagery (KI) paradigms. Then, to visual and kinesthetic stimuli to the user to convey VI, MI and/or
kinesthetic imagery (KI).
EEG events are decoded as desired behavior to the robot by
a Finite State Machine (FSM), afterwards converted them into
provide mechanisms to close the loop with the robot through the
desired trajectories to the robot controller by using kinematic
robot control, in contrast to previous proposals, wherein robot
maps based on differential kinematics.
was assumed implicitly an kinematic external agent out of the
In this paper, we propose to use coherently several tools from loop. In our proposal the EEG signals acquisition and process-
advanced robotics to study the problem on controlling a dy- ing block is enclosed into the BCI subsystem, whose output is
namic robot with low workload using an EEG-based BCI. The handled by the FSM within the “Robotic Tools” subsystem, to-
result is an interface to control a robot that purposely behaves gether with the control guarantees stability properties according
as a dynamic one using a low cost EEG device. In contrast, to user specifications. The BCRI system is conformed by five
when the robot behaves noticeably as kinematic one, workload main stages, shown in Fig. 1.b, which includes:
increases up to the point that user fails to yield useful EEG
User: It is assumed healthy adult subjects that are able to
commands. That is, when we consider a dynamic robot as the
produce voluntarily a given brain signal under training based
external agent, the proposed Brain Computer Robot Interface
on the paradigm of VI, MI, or KI.
(BCRI), that complies to physical and real time constraints
within the time-base of human user, outperforms in comparison EEG signals acquisition and processing: The Emotiv EPOC
TM

to considering a kinematic robot. is the EEG data acquisition system (14 electrodes under 10 − 10

157
IFAC SYROCO 2015
August 26-28, 2015. Salvador, BA, Brazil U. Sanchez-Fraire et al. / IFAC-PapersOnLine 48-19 (2015) 154–159
156

protocol, with a sampling rate of 128 Hz). The SDK decodes Robot: It could be a real or virtual robot, either kinematic or
raw data, signal processing with proprietary algorithms to pro- dynamic, or even-based in the continuous, sampled or discrete
vide EEG command as well as cognitive mental states under domain.
visual and motor imagery after training.
Haptic Device: Kinesthetic display can be rendered with a hap-
Finite State Machine (FSM): It processes the EEG-based tic device (flat motor, tactual, haptic or vibrotactile stimulus)
commands and cognitive user states. Two FSM are proposed: to reinforce a given motor paradigm or convey proprioceptive
a) Automatic FSM process commands under conditional user sense of the task or to display kinesthetic imagery. It computes
action to establish desired robot motion and computes desired the desired force fd to be displayed by the robotic haptic device.
robot trajectories and supervise task completion, see fig. 2; and It our case, it is considered a haptic commercial device such as
b) Semiautomatic FSM similar to automatic one but it does the FALCON R
, that is controlled in open architecture through
not contain the conditional confirmation of the user such that f¯d = J (qd )τc , where J −T (qd ) is the inverse transpose of the
−T
it carries out an environment analysis of the agent, including robot Jacobian and τc stands for the control torques to display
finite time stability, see fig. 3. fd with that torque controller.

3. THE DYNAMIC EXTERNAL AGENT/ROBOT AND

CONTROL DESIGN

For completeness, we reviewed well-known robot modeling

and control schemes, for kinematic and dynamic models, in-
cluding stability issues.

3.1 Kinematic Robot

Typically, the external agent has been considered as a kinematic

entity, either virtual (a pointer to pick a letter, a word, or an ob-
ject, or a visual mark, in the screen, or to produce an audio stim-
uli with a speech synthesized, or a robot linkage mechanism) or
real (a robot manipulator, a wheelchair, or any physical entity
Fig. 2. State flow of the automatic FSM. without dynamical attributes). In a kinematic agent, inertia is
not considered, neither any dynamic effects, (including friction)
Robotic Tools: This subsystem provides modeling and control as if it were composed only of geometric attributes. In this case,
algorithms for signal conditioning and motion planning, as the robot is modeled only by the geometric relationships of
well as dynamic simulation, solvers, and validation of a given generalized coordinates with respect to a frame, and their time
behavior. These tools are tailored to the performance of the variations. Here, we briefly recall it, and discuss its limitations.
robot and to compute the user stimuli based on the imagery i
paradigm used. Contains the spatio-temporal FSM, the environ- Kinematic Model Consider the homogeneous matrix Hi−1
ment analysis, kinematic maps and singularity analysis, to deal that relates adjacent frames i − 1, i based on the Denavit-
with spatio-temporal constraints imposed by the tasks and/or Hartenbergh. Forward kinematics and its differential mappings
the user, as well as the calculation of the visual and force stimuli can be obtained of the differential drive mobile robot.
to the user.
Kinematic Control For regulation consider the well-know PD
control.

3.2 Dynamic Robot

When the external agent considers the forces that causes mo-
tion, a well-known modeling formalism based on energy bal-
ance methods provides closed-form differential models, whose
input is the external controller. In closed-loop, the external
controller modifies the dynamics and sets a desired dynamic
behavior.

Dynamic Model Consider the dynamic model of a rigid

serial n-link robot manipulator with all revolute joints given
by the well-known Euler-Lagrange equation, that is H(q)q̈ +
C(q, q̇)q̇ + g(q) = τ − τ f + τd where q ∈ ℜn , q̇ ∈ ℜn are the gen-
eralized position and velocity joint coordinates, respectively,
H(q) ∈ ℜn×n denotes a symmetric positive definite inertial
matrix, C(q, q̇) ∈ ℜn×n represents the Coriolis and centripetal
forces, g(q) ∈ ℜn models the gravity forces, and τ ∈ ℜn stands
for the torque input. Term τ f = B0 q̇ + B1 tanh(ν q̇) stands for
joint friction, B0 , B1 are positive definite n × n matrices mod-
Fig. 3. State flow of the semiautomatic spatio-temporal FSM. eling the viscous damping, and the dry friction, and ν its the

158
 IFAC SYROCO 2015
August 26-28, 2015. Salvador, BA, Brazil U. Sanchez-Fraire et al. / IFAC-PapersOnLine 48-19 (2015) 154–159 157

approximation coefficient, respectively. Disturbance torque τd corner), but now the “goal” mark is the starting point. The
is assumed bounded and constant. spatio-temporal constraints are handled by the semiauto-
matic FSM described in Fig. 3, while a second task with
Dynamic Control We have considered a high-end dynamic no semiautomatic FSM is also considered. The task fin-
controller that guarantees tracking under unmodeled dynamics ishes when the robot reach the goal point or when the user
and uncertainties, Parra-Vega (2003). This controller is based gives up on the task. The spatio-temporal semiautomatic
on high order sliding modes (SM2) and a time-base generator FSM commands the robot in the desired direction until the
(TBG), that lead to a smooth, fast and precise motion with user commands otherwise. The FSM maps the cognitive
spatial and time constraints, for smooth desired trajectories that mental state from the BCI in a desired task as shown, with
are conditioned with the FSM. Worthily to say, the TBG allows measurements as experiment A:
to guarantee well-posed finite-time convergence to meet user · PUSH: move agent in X of local frame.
expectations within time constraints. · PULL: move agent in Y of local frame.
· LIFT: rotate agent respect to Z axis of local frame.
4. MATERIAL AND METHODS • Experiment D: Commanding a mobile robot with and
without kinesthetic imagery. The task is shown in Fig. 4,
Subjects Three right-handed subjects (males), ranged in age to be carried out under bimodal stimuli, afterwards with-
from 25 to 27 years (mean age 26 years) unfamiliar with a out it. The kinesthetic (haptic) interface conveys visual
BCI system. All volunteers meet inclusion criteria and sign a and vibrotactile stimuli. The visual displays the 3D virtual
declaring written informed consent. The subjects reported no state of the robot and the vibrotactile force is calculated
β
history of neurological or psychiatric disorders and were taking by f = α + 1+e X
Hz
no medications at the time of the study. where α is the minimal desired frequency, β is the
maximum desired
 frequency and
EEG data acquisition and System EEG were recorded with
a 14-channel electrode device based on the 10-10 electrode- eX  = (x − xd )2 + (y − yd )2 + (θ − θd )2 such that
placement system (Emotiv Epoc, Australia). The position of the f ≤ (α + β ) for eX  = 0, that is frequency of the vi-
EEG sensors were AF3, AF4, F7, F3, F4, F8, FC5, FC6, T7, T8, brotactile is inversely proportional to the position error,
P7, P8, O1 and O2, using P3 and P4 at as reference locations. then vibrating at greater frequency occurs when the user
Two robots are considered for our experiments: the omnidirec- changes to the next cognitive mental state (to change the
TM
tional mobile robot Robotino by Festo, and an experimen- motion direction of robot). This is used to convey spatial
tal 1-DoF dynamic robot arm subject to gravity and friction. proprioception of the task. The frustration level of the
A dynamic simulator is programmed including a display and user, the spectral powers of the cognitive mental states and
menu for set-up and tuning gains. A high-end PC equipped with the total time of the task are measured.
graphic card is used in a C++ programming environment. • Experiment E: Commanding a robot arm. A simple yet
nonlinear robotic arm of 1DoF is modeled as a nonlinear
Five Behavioural Tasks and Experimental Protocol pendulum and commanded to reach a desired orientation
at the unstable equilibrium in θd = 90◦ under spatio-
• Experiment A: Training cognitive state under different
temporal FSM and a TBG controller. In this case, the FSM
types of mental imagery. The subjects are trained to
interprets the user cognitive states as follow:
produce a specific mental cognitive state under specific
· PUSH: rotate the agent clockwise.
stimuli that triggers the neuroscience paradigm. Three
· PULL: rotate the agent counter-clockwise.
minutes later, subjects are directed to reproduce the mental
cognitive state without any stimuli, i.e.. The following
imagery are considered:
· VI. A flashing red/yellow color screen is presented to
the user as visual stimuli.
· KI. A circular motion stimuli exerts a contact force
with a haptic device.
· VI and KI. Both given above.
and the following measurements are recorded:
· Spectral power
· Command activation time Time interval that users Fig. 4. Desired task with an automatic FSM
maintain active a cognitive state.
· Frustration of the user. A cognitive state obtained by 5. RESULTS
a proprietary (patented) method of the BCI headset.
• Experiment B: Commanding a mobile robot under Imagery Paradigm: Experiment A. Experiment A results
an automatic FSM under different controllers. The show that the frustration level is low when using any of imagery
automatic FSM of Fig. 2 is used to complete the task of paradigms, but time lapse and spectral power are bigger when
Fig. 4. One cognitive command triggers the task wherein using both paradigms, as shown in Table 1. The command
the FSM manages the flow until it is finished, using two remains active for long period by producing an stable cognitive
controllers, the integral sliding mode and the TBG, to state, as shown in Fig. 5. Thus, EEG command is improved
assess the L2 of tracking error. when both VI and MI are used.
• Experiment C: Commanding a mobile robot without
a FSM, then with a FSM. The task is lead out the robot Controller: Experiment B Trajectory errors using TBG and
from scenario of Fig. 4 (note the “exit” in the upper right SM2 controllers for experiment B are shown in fig. 6. Both

159
IFAC SYROCO 2015
August 26-28, 2015. Salvador, BA, Brazil U. Sanchez-Fraire et al. / IFAC-PapersOnLine 48-19 (2015) 154–159
158

Table 1. Results of experiment A. Comparison between dis- (A) Robot Trajectory (B) Robot Trajectory
2.5 3.5
tinct imagery training, where SP represents the mean spectral 2 3

power, te stands for the total time period that cognitive state is 1.5
2.5

active, and F is the mean of the frustration level. 2

Y [m]

Y [m]
1
1.5
0.5
1

Command attributes 0 0.5

User Paradigm SP te F −0.5

−2 −1 0
X [m]
1 2 3
0
−2 −1 0 1
X [m]
2 3 4

1 Visual 0.5939 13.6032 0.3163 (A) Robot Orientation (B) Robot Orientation

2 Motor 0.4556 40.3530 0.9995

1000 500

800 400
3 Both 0.6416 50.2255 0.3707

Orientation [deg]

Orientation [deg]
600 300

PUSH Visual Imagery PUSH Motor Imagery PUSH Bimodal 400 200
1 1 1

200 100
0.8 0.8 0.8

0 0
0 100 200 300 400 500 600 700 0 200 400 600 800
0.6 0.6 0.6 Time [seg] Time [seg]
Power

Power

Power
0.4 0.4 0.4

0.2 0.2 0.2

Fig. 7. Robot trajectories: (A) without a FSM, (B) with a spatio-temporal
0
0 10 20 30 40
0
0 10 20 30 40 50
0
0 10 20 30 40 FSM.
t [seg] t [seg] t [seg]

(A) Frustration (B) Frustration

1 1

Fig. 5. Experiment A: Spectral power (SP) for three imagery paradigms. Each 0.9 0.9

0.8
plot represent the SP of a generated command over time, where 0 SP
0.8

0.7 0.7

means that the command was not generated and 1 SP means that the 0.6 0.6

Power

Power
command completely identified. 0.5 0.5

0.4 0.4

0.3 0.3

controllers gives smoothness to the trajectory, but TBG control 0.2 0.2

fulfills time constraints if an adequate time tb is imposed. As 0.1

0
0.1

result, the TBG controller outperforms because it leads to a

0 100 200 300 400 500 600 0 100 200 300 400 500 600 700
Time [seg] Time [seg]

stable and smooth motion, in addition to allow setting desired

finite time that regulates speed to an acceptable level. Fig. 8. User frustration levels: (A) Without spatio-temporal FSM and, (B)
with spatio-temporal FSM. It is clear that the user frustration level is
TBG SM2 lower when commanding a robot under the FSM management.

2
Position Trajectory Error
2
Position Trajectory Error
may lead to a false positives as shown in Fig. 9. In plot (A),
1 1 ”lift” activation is considered from false positives, while in plot
Error X [m]

Error X [m]

0 0
(B) such false positives are suppressed.
−1 −1

−2 −2
0 10 20 30 40 50 60 70 80 0 20 40 60 80 100 120
t [seg] t [seg] (A) PUSH (B) PUSH
1 1
4 4

0.8 0.8
Error Y [m]

Error Y [m]

2 2
0.6 0.6
Power

Power

0 0
0.4 0.4
−2 −2
0 10 20 30 40 50 60 70 80 0 20 40 60 80 100 120
0.2 0.2
t [seg] t [seg]

50 0 0 0
0 50 100 150 200 250 0 50 100 150 200 250
Error PSI [deg]

Error PSI [deg]

0
−50
(A) PULL (B) PULL
1 1
−50

0.8 0.8
−100 −100
0 10 20 30 40 50 60 70 80 0 20 40 60 80 100 120
t [seg] t [seg] 0.6 0.6
Power

Power

0.4 0.4

0.2 0.2
Fig. 6. Position error of task corresponding to Fig. 4 using a TBG and SM2
0 0
contollers. 0 50 100 150 200 250 0 50 100 150 200 250

(A) LIFT (B) LIFT

1 1
The spatio-temporal FSM: Experiment C The trajectories 0.8 0.8
that were seeking to take the robot out are shown in Fig. 7. 0.6 0.6
Power

Power

Using the spatio-temporal FSM results in a decrease of the 0.4 0.4

frustration level as seen in Fig 8, despite the task was not com- 0.2 0.2
pleted in neither case. We conjecture that user failed the task 0 0
due increasing frustration provoked by false positives detected 0 50 100 150 200 250 0 50 100 150 200 250

by the BCI, causing undesired motion on the robot. We con-

firmed that conjecture by repeating experiment C connecting a
cognitive state emulator to the dynamic simulator. Doing so, the Fig. 9. Spectral powers of commands (A) without bimodal imagery, and (B)
task was completed and we confirm that our FSM system works under bimodal imagery (where false positives of ”lift” are suppressed).
as expected, given the right commands.

Bimodal Imagery: Experiment D It is already known that Commanding the 1-DoF Robot: Experiment E Frustration
a bimodal imagery enhances EEG signals, Thurlings (2012), level increases after user attempt to reach the exact point of the
which in these experiments is obtained with haptic and visual desired position, see Fig 10, causing oscillation near θd = 90.
imagery. Notice that a vibrotactile stimuli prevents patterns that This exemplifies a real behavior when we try to balance a stick

160
 IFAC SYROCO 2015
August 26-28, 2015. Salvador, BA, Brazil U. Sanchez-Fraire et al. / IFAC-PapersOnLine 48-19 (2015) 154–159 159

120
Robot Orientation
1
Frustration McFarland, D., and Wolpaw, J. (2008). Brain-Computer
100
0.9 Interface Operation of Robotic and Prosthetic Devices.
80
0.8

0.7
Computer-IEEE Computer Society, pp. 52-56.
Millan J. R., Renkens, F., Mourino, J., and Gerstner, W. (2004).
Orientation [deg]

0.6

Power
Noninvasive brainactuated control of a mobile robot by hu-
60 0.5

0.4
40
0.3 man EEG. IEEE Trans Biomedical Enggr., vol.51, pp. 1026-
1033.
0.2
20
0.1

0
0 20 40 60 80 100 120 140
0
0 20 40 60
Time
80 100 120 140 Neuper, C., Scherer, R., Reiner, M., and Pfurscheller, G. (2005).
Imagery of motor actions: Differential effects of kinesthetic
Time [seg]

Fig. 10. Trajectory of robotic arm with θd = 90, and frustration level of and visual-motor mode of imagery in single trial EEG. Cog-
experiment E. The frustration level rises as user could not keep the nitive Brain Research, vol. 25, pp. 668-677.
desired position due the overload that the task implies. Nicolelis, M. and Chapin, J. (2002). Controling robots with the
mind. Scientific American, vol. 287, pp. 46-53.
in an upright position, suggesting a good matching with mental Patil, P. and Turner, D. (2008). The Development of Brain-
models. Machine Interface Neuroprosthetic Devices. Neurotherapeu-
tics: The Journal of the American Society for Experimental
REFERENCES NeuroTherapeutics, vol. 5, pp. 137-146.
Parra-Vega, V., Arimoto, S., Liu, Y., Hirzinger, G. and Akella,
Bakardjian, H. (2010). Optimization of Steady-State Visual Re- P. (2003). Dynamic Sliding PID Control for Tracking of
sponses for Robust Brain-Computer Interfaces. Tokyo Univ. Robot Manipulators: Theory and Experiments. IEEE Trans.
Birbaumer, N., Ghanayim, N., Hinterberger, T. I. Iversen, Robotics and Automation, vol. 19, No. 6, pp. 967-974.
Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E. and Rebsamen, B., Zhang, H., Wang, C., Teo, C. and Burdet,
Flor, H. (1999). A spelling device for the paralysed, Nature, E.(2011) A Brain Controlled Wheelchair to Navigate in Fa-
vol. 398, pp. 297-298. miliar Environments. IEEE Transactions on Neural Systems
Bogue, R. (2010). Brain-computer interfaces: control by and Rehabilitation Engineering, vol. 18, No. 6, pp. 590-598.
thought. Industrial Robot: An International Journal, ISSN Ron-Angevin, R., Lopez, M. and Pelayo, F. (2009). The Train-
0143-991X, pp. 126-132. ing Issue in Brain-Computer Interface: A Multi-disciplinary
Brown, L. M., Brewster, S. A., and Purchase, H. C. (2005). Field. IWANN 2009, Part I, LNCS 5517, Springer-Verlag, pp.
A first investigation into the effectiveness of tactons. Pro- 666-673.
ceedings of the First Joint Eurohaptics Conference and Sym- Stinear, C.M., Byblow,W. D., Steyvers,M., Levin, O., and
posium on Haptic Interfaces for Virtual Environment and Swinnen, S. P. (2006). Kinesthetic, but not visual, motor
Teleoperator Systems (WHC05), pp. 167-176. imagery modulates corticomotor excitability. Experimental
Brunner, P., Bianchi, L., Guger, C., Cincotti, F. and Schalk, G. Brain Research, vol. 168, pp. 157164.
(2011). Current trends in hardware and software for brain- Thurlings, M., Brouwer, A., Van, J., Blankertz, B., Werkhoven,
computer interfaces (BCIs). Neural Engineering, 8 025001, P., (2012). Does bimodal stimulus presentation increase ERP
IOP Publishing, vol. 8, pp. 1-7. components usable in BCIs?. J. Neural Engineering., vol. 9,
Cheng, M., Gao, X., Gao, S. and Xu, D. (2002), Design and pp. 1-15.
Implementation of a Brain-Computer Interface With High Vidal, J. J. (1993). Toward direct brain-computer communica-
Transfer Rates. IEEE Transactions on Biomedical Engineer- tion. Annual Review of Biophysics and Bioengineering, vol.
ing, vol. 49, no. 10. 2, pp. 15780.
Curran, E. and Stokes, M. (2003). Learning to control brain Wolpaw, J. R., Birbaumer, N., William, J., Quatrano, D., Robin-
activity: A review of the production and control of eeg com- son, C. and Vaughan,T. (2000). BrainComputer Interface
ponents for driving brain-computer interface (BCI) systems. Technology: A Review of the First International Meeting.
Brain and Cognition, vol. 51, pp. 326-336. IEEE Trans. on Rehabilitation Engineering, vol. 8, no. 2.
Decety, J. (1996). The neurophysiological basis of motor im- Wolpaw, J. R., Birbaumer, N., William, J., Quatrano, D., Robin-
agery. Behavioral Brain Reseach, vol. 77, pp. 45-52. son, C. and Vaughan,T. (2002). BrainComputer Interface for
Dechent, P., Merboldt, K., Frahm, J. (2004). Is the human communication and control. Neurophysiology, vol. 113, no.
primary motor cortex involved in motor imagery?. Research 6, pp. 767-791.
report, Cognitive brain research, Elsevier, pp. 138-144. Wolpert, D.M, Ghahramani, Z. and Jordan, M. I. (1995). An
Felzer, O. (2001). On the Possibility of Devel- Internal Model for Sensorimotor Integration. Science, New
oping a Brain-Computer Interface (BCI) Tech- Series, vol. 269, no. 5232, pp. 1880-1882.
nical Report, URL: http://www.st.informatik.tu- Xiao, J., Padoa-Schioppa, C. and Bizz, E. (2006). Neuronal
darmstadt.de:8080/felzer/eeg.pdf. Correlates of Kinematics-to-Dynamics Transformation in the
Kosslyn, S., Ganis, G. and Thompson, W. (2001). Neural foun- Supplementary Motor Area. Experimental Brain Research.
dations of Imagery. Nature Reviews, Neuroscience, vol. 2,
pp. 635-642.
Lupien, S.J., Maheu, F., Tu, M., Fiocco, A. and Schramek, T. E.
(2007). The effects of stress and stress hormones on human
cognition: Implications for the field of brain and cognition.
Brain and Cognition, vol. 65, pp. 209-237.
Maye, A., Zhang, D., Wang, Y., Gao, S., Engel, A. (2011).
Multimodal Brain-Computer Interfaces. Tsinghua Science
and Technology, ISSN 1007-0214, vol. 16, no. 2, pp. 133-
139.

161