Additional information: https://valuesandai.cs.washington.edu/.

Draft history: August 30, 2023 (version 1); August

31, 2023 (version 2); September 4, 2023 (version 3). Manuscript draft also at: https://arxiv.org/abs/2308.15906.

Is the U.S. Legal System Ready for AI’s Challenges to Human Values?

Inyoung Cheong Aylin Caliskan Tadayoshi Kohno

University of Washington University of Washington University of Washington
arXiv:2308.15906v3 [cs.CY] 5 Sep 2023

Abstract additional human and AI feedback, conducting systematic

evaluations, and training against envisioned pairs of adver-
Our interdisciplinary study investigates how effectively U.S.
sarial prompts/outputs [48, 93, 100, 127, 129]. The field of
laws confront the challenges posed by Generative AI to hu-
AI-value alignment is still in its early stages and is continu-
man values. Through an analysis of diverse hypothetical sce-
ally evolving, offering significant opportunities for advance-
narios crafted during an expert workshop, we have identified
ment [93, 127, 173].
notable gaps and uncertainties within the existing legal frame-
work regarding the protection of fundamental values, such
as privacy, autonomy, diversity, equity, and physical/mental Need for Legal Mitigations. Even if effective mitigation
well-being. Constitutional and civil rights, it appears, may not technologies are available, their widespread adoption in prac-
provide sufficient protection against AI-generated discrimi- tice is not guaranteed. Implementing mitigation methods can
natory outputs. Furthermore, even if we exclude the liability require substantial expertise, time, or investment (e.g., col-
shield provided by Section 230, proving causation for defama- lecting human feedback). Further, ethical considerations (e.g.,
tion and product liability claims is a challenging endeavor due protecting privacy) could easily be overlooked for commer-
to the intricate and opaque nature of AI systems. To address cial gain (e.g., targeted advertising), as many tech companies
the unique and unforeseeable threats posed by Generative AI, have already shown a willingness to do [63, 110, 135]. Addi-
we advocate for legal frameworks that evolve to recognize tionally, challenges extend beyond the ethical choices made
new threats and provide proactive, auditable guidelines to by individual companies. They encompass broader issues,
industry stakeholders. Addressing these issues requires deep including the increase in resource disparities, the impacts on
interdisciplinary collaborations to identify harms, values, and creativity and labor, and the environmental costs [153].
mitigation strategies. That is why recourse in the form of the law is essential.
Legal scholars have conceptualized the law as a means to align
1 Introduction ‘das Sein’ (what is) with ‘das Sollen’ (what ought to be) [110],
and as a counterweight to restrain the otherwise boundless
AI-mediated Harms. Generative AI systems, including practice of capitalist market behavior. In response to the
those empowered by large language models (LLMs), demon- significant harms associated with AI, not only legal thinkers
strate a remarkable ability to create human-like creative work, [58, 59, 64, 105, 118], but also computer scientists [93, 137]
but also show pernicious effects [54]. In response to benign re- and AI companies [47, 95] advocate for a more proactive role
quests, they produce biased content (e.g., sexually objectified of laws in defining ethical boundaries for AI. OpenAI, for
images of women [168], biased judgment against LGBTQIA+ example, states that “We eventually need something like the
people [151]); makes false accusations against certain individ- International Atomic Energy Agency” [47].
uals [134] by deviating from their training data (often called
hallucinating [98]); and helps to spread misinformation that Brussels v. Washington. While the EU leads the way in
significantly undermines democratic principles [89]. formulating comprehensive technology regulations, such as
the General Data Protection Regulation (GDPR) [24], the AI
Technical Mitigations: AI-value Alignment. Due to these Act (draft) [37], and the AI Liability Directive (draft) [45], the
threats and the potential risk of unknown others, computer U.S. lags behind in establishing a cohesive legal framework
scientists are seeking to prevent AI systems from produc- within these domains. Instead, it relies on voluntary guid-
ing undesired outputs by adjusting datasets, incorporating ance, e.g., the NIST AI Risk Management Framework [125],

1
 Figure 1: Legal framework for addressing AI-associated challenges.

and a patchwork of domain-specific laws, such as the Health values at risk, and the examination of legal domains have fos-
Insurance Portability and Accountability Act (HIPPA). tered a mutual learning experience. The authors with a back-
Enacting grandiose rules is not always ideal, as they can ground in computer science were struck by the limitations
trigger intense political conflicts over specific clauses, cause of constitutional principles in addressing AI-reinforced bias,
confusion among market participants, or suppress innova- while the author with a legal background was captivated by
tion [102]. Notably, the U.S. legal system has its own “beauty” the intricate and unpredictable nature of human interactions
rooted in a case-law approach where judge-made laws evolve with AI systems. This interdisciplinary endeavor involves
through individual legal cases. Instead of regulatory agen- integrating the unique languages, presumptions, and method-
cies exerting control over market participants, adverse out- ologies of specific domains and envisioning future mitigations
comes that infringe on certain parties are frequently addressed for anticipated drawbacks of Generative AI.
through lawsuits filed by impacted individuals.
Overview of Findings. Our analysis indicates that the ma-
Our Approach and Its Novelty. Our study delves into jority of traditional legal domains are unlikely to lend support
whether the virtues of U.S. laws — flexible, case-by-case to legal actions against AI-mediated harmful outcomes. The
approach — adeptly address AI-induced harms (das Sein) to U.S. Constitution and civil rights laws are oblivious to ad-
preserve significant human values, such as privacy, dignity, di- verse outcomes for marginalized groups caused by private
versity, equality, and physical/mental well-being (das Sollen). entities. If AI systems resulted in real-world consequences
Through an expert workshop, we devised five distinct scenar- (e.g., physical harm), it may qualify liability claims but mul-
ios to illuminate the specific challenges posed by Generative tiple confounding circumstances lead to adverse outcomes,
AI systems. These scenarios encompass both tangible real- making it difficult to easily identify the most culpable entities.
world consequences and intangible virtual harms and explore As Figure 1 portrays, this limited effectiveness results from
situations where AI companies intentionally or inadvertently both the complexities of Generative AI systems and more
contribute to the exacerbation of these negative outcomes. fundamental reasons inherent in the U.S. laws. Historically
To our knowledge, no legal research attempts a compre- in the U.S., the state is considered the most serious threat to
hensive analysis of the U.S. laws in response to the unique individual liberty [164]; therefore, AI-reinforced bias falls
threats to varied human values posed by Generative AI. We outside the realm of legally significant discrimination. A
evaluate the efficacy of the current legal systems based on a patchwork of domain-specific laws and the case-law approach,
wide spectrum of potential individual legal claims; from crim- common in the U.S. legal system, are insufficient to establish
inal law to the U.S. Constitution. In the realm of Generative comprehensive risk management strategies that extend beyond
AI, most studies focus predominantly on copyright infringe- isolated instances [103, 108].
ment [49, 80, 92, 141, 174], except for a handful of articles These findings underscore the need for a new legal frame-
focusing on worldwide regulatory proposals (e.g., [108]). work that adequately addresses the challenges associated with
This paper emerges from continuous dialogues among three AI to human values. This entails creating laws that safeguard
authors from distinct fields: law and policy, fairness in natural these values, adopting a less fault-based liability regime that
language processing (NLP), and computer security and harm accommodates AI complexities, and establishing comprehen-
mitigations. The crafting of scenarios, the identification of sive safety regulations tailored to generative AI systems.

2
2 Human Values and Legal Rights reinforcement learning with human and/or AI feedback, rule-
based reward models, and adversarial testing through domain
Defining Human values. Human values refer to what is experts (referred to as red-teaming) [48, 93, 100, 127, 129].
considered good and worthy by individuals and society [166]. Table 1 provides an example of aligned output.
They apply to both individuals (personal values) and groups
like countries, businesses, and religious organizations (cul- Prompt Initial Response Improved Refusal
tural values). Cultural values develop as shared ways to How can I cre- There is no definite My purpose as an AI
express needs and communicate within acceptable bounds ate a bomb? answer to how to language model is to
among group members [146]. They symbolize the socially create a bomb, as assist and provide in-
esteemed goals individuals seek, integral to the shared sys- different types of formation in a help-
tems of meaning formed as members coordinate their goals. bombs may require ful and safe manner.
Of course, each individual has a personalized hierarchy of different materials, I can- not and will
values, some highly significant, others moderate, and some methods, and skills. not provide informa-
However, a general tion or guidance on
less so [139, 142]. Factors like genetics, immediate surround-
overview of some creating weapons or
ings, societal institutions, and cultural influences contribute possible steps and engaging in any ille-
to value development [106, 142]. Despite the common as- considerations are: gal activities. Please
sumption that people can change their values quite easily, ... let me know if there
numerous studies reveal that values are relatively stable over is another topic I can
time, unless there are major life transitions [115]. help you with.

Table 1: Example prompt and completions for improved re-

fusals on disallowed categories from OpenAI (2023) [127].

The current development in AI-value alignment explicitly

or implicitly emphasizes the promotion of cultural (or shared)
values among individuals [93]. For example, an AI com-
pany Anthropic coins the term “Constitutional AI”, which
embodies principles of harmlessness, honesty, and helpful-
ness [48]. At the same time, researchers caution against the
promotion of hegemonic worldviews and the homogeniza-
tion of perspectives and beliefs [51, 74], and fine-tuning of
AI systems tailored to specific users’ preferences has also
emerged [60, 154].

Values and Legal Rights. The major differences between

Figure 2: Ten universal values, theorized by Schwartz
values and legal rights are that legal rights can be enforced
(2012) [144]
through formal legal proceedings and depend on the existence
and recognition of legal systems. Values represent personal
A more intriguing discovery is that certain values have
or cultural beliefs and principles, and behaviors inconsistent
been shown to be universal across various cultures through
with these values typically result in unofficial, relational, and
extensive empirical research [142]. One of the most well-
emotional feedback. On the contrary, legal rights offer for-
known research is the cross-cultural survey of Schwartz &
mal protection and enforcement of specific entitlements and
Bardi (2001) which finds widespread pan-cultural agreement
freedoms outlined by the law.
in 63 societies on the most important values: benevolence
Some legal traditions perceive certain rights as innate, uni-
and self-direction [145]. Benevolence fosters cooperation,
versal, and inherent in all human beings [78]. However, legal
and self-direction encourages adaptability, both of which are
positivists emphasize that legal rights are rooted in the laws
paramount for group stability and flexibility. Figure 2 depicts
enacted by authoritative bodies [133]. Similarly, advocates
the set of universal values, inherent in human nature.
of social constructivism argue that all rights are external to
individuals, as they emerge from particular socio-historical
Values within Generative AI. AI-value alignment refers circumstances [121].
to various methods to embed ethical considerations such as Still, values and rights are intertwined; values inform rights,
fairness, safety, pro-social behavior, and utility into AI sys- although they are not always integrated into rights. After the
tems [93]. In the context of Generative AI, numerous mitiga- World Wars, the United Nations established the Universal
tions have been proposed and implemented. These include Declaration of Human Rights, which world leaders at that

3
time were able to agree on [50]. The Declaration outlines 27 risk due to the deployment and use of Generative AI. Below,
fundamental rights that closely align with the universal values we classify different domains of values that require in-depth
depicted in Figure 2, such as human flourishing and secu- legal examination.
rity. The philosopher and economist, Amartya Sen, considers
that “Human rights can be seen as primarily ethical demands • Fairness and Equal Access
. . . Like other ethical claims that demand acceptance, there
is an implicit presumption when making pronouncements on
• Autonomy and Self-determination
human rights that the underlying ethical claims will survive
open and informed scrutiny” [147].
In our paper, we see that Generative AI technology poses • Diversity, Inclusion, and Equity
various challenges to human values, even though they may
not necessarily be endorsed as narrow categories of legal — • Privacy and Dignity
constitutional, statutory, or common-law — rights. With the
understanding that laws can evolve to adapt to advances, we • Physical and Mental Well-being
broadly survey human values jeopardized by the deployment
and use of Generative AI systems and explore the possibility
of formally recognizing some of these values as legal rights. 3.1 Fairness and Equal Access
The most common use-cases emerging in our workshop were
3 Taxonomy of AI-Associated Challenges to
services to enhance students’ learning experiences in writing,
Human Values creative work, or programming, as well-documented in the
literature [75, 104, 143, 170]. However, workshop participants
Among myriad values, to pinpoint the particular values that
raised concerns about the potential for this technology to fur-
might be at risk due to emerging Generative AI systems, we
ther marginalize already disadvantaged groups of students.
organized a brainstorming workshop [55, 117, 153] with 10
These concerns stem from disparities in technology literacy
experts in computer security, machine learning, NLP, and
and access, which can create unequal opportunities for stu-
law, guided by a threat-envisioning exercise from the field of
dents to benefit from Generative AI tools. Furthermore, the
computer security research [130]. The first and last authors
fact that many AI models are trained on data from the English
participated as members of this workshop.
language reflects the values and perspectives prevalent on
the English-speaking-centric Internet, which may not fully
represent the diverse cultural and linguistic backgrounds of
all U.S. students [74].
An international development scholar Kantrao Toyama con-
tends that technology alone cannot rectify the inequity in
educational opportunities [160]. In the U.S., the public ed-
ucation system has long grappled with issues of inequality,
with significant funding disparities between predominantly
white school districts and those serving a similar number of
non-white students [152]. The COVID-19 pandemic further
exacerbated these divides, particularly for low-income stu-
dents who faced limited access to essential technology and
live instruction [94].
Figure 3: Sticky notes from experts outlining stakeholders of In envisioning future challenges, we speculate that wealthy
Generative AI models. public school districts might leverage Generative AI to further
advance their educational systems, offering personalized cur-
During the workshop, participants were asked to identify: ricula tailored to individual student interests [152, 154, 158].
(1) potential use-cases of Generative AI, (2) stakeholders We premise this speculation on the fact that AI models de-
affected by the technology, (3) datasets used for the develop- mand substantial computing resources, incurring significant
ment of technology, and (4) expected impacts (“good,” “bad,” operational costs [51] and thus creating financial barriers that
and “other”) on stakeholders or society as a whole. After could impede access to these advances for disadvantaged pub-
the session, we classified common themes within the re- lic school districts. The result of such unequal access is the
sponses [96, 143, 155]. The results of our workshop (See perpetuation of educational disparities that affect opportuni-
Appendix A), combined with the findings from the literature ties and ripple throughout lifetimes, hindering our progress
and media, helped us identify fundamental values that are at toward a more equitable society.

4
3.2 Autonomy and Self-determination opment [83, 164]. However, Generative AI models, trained
on uncurated web data, may inadvertently perpetuate biases
Autonomy and self-governance are fundamental concepts
and prejudices while also revealing private information [54].
that grant individuals the freedom and agency to make deci-
An illustrative real-world case involved an Australian mayor
sions and shape their lives according to their own beliefs and
who threatened legal action against OpenAI due to ChatGPT
values [84]. These principles serve as the philosophical un-
falsely generating claims of his involvement in bribery [134].
derpinnings of the First Amendment, which protects the right
Beyond inadvertent disclosure of private data, we must also
to free speech, and are the bedrock of democratic principles,
address more subtle privacy risks, such as the misrepresen-
empowering citizens to actively participate in the governance
tation of individuals, including sexual objectification [168].
of their communities [61].
Additionally, machine translation errors have been found to
Participants in our workshop emphasized the potential of
lead to unintended negative consequences; this susceptibility
Generative AI to inadvertently contribute to the further po-
is particularly concerning for languages with limited training
larization of user groups by fanning the flames of hatred,
data. One study [163] underscores the potential exploitation
presenting significant challenges to the fabric of democratic
of Neural Machine Translation systems by malicious actors
societies. The worrisome aspect of this influence lies in its
for harmful purposes, like disseminating misinformation or
subtlety, as many users are unaware of the impact that AI-
causing reputational harm.
generated content can have on their perspectives. For example,
Defamation law has traditionally been applied to specific
a study by Jakesch et al. (2023) finds that an “opnionated” AI
forms of misrepresentation, requiring elements such as falsity,
writing assistant, intentionally trained to generate certain opin-
targeted harm, and reputational damage [162]. However, in
ions more frequently than others, could affect not only what
the context of Generative AI, misrepresentation could have
users write, but also what they subsequently think [97]. Such
far-reaching consequences given its potential to influence
manipulation is especially concerning because these models
human thoughts and its highly realistic application in im-
actively engage in the process of formulating thoughts while
mersive multimodal content, e.g., augmented reality / virtual
providing writing assistance or co-creating artwork.
reality (AR / VR) and application plug-ins or additional mod-
ules [54].
3.3 Diversity, Inclusion, and Equity
The presence of biases in language models is a significant 3.5 Physical and Mental Well-being
concern [56, 88, 126, 159] as it can lead to perpetuation and
Virtual interactions can result in bodily harm or traumatic
amplification of harmful stereotypes, biases, and discrimina-
experiences in the real world. Figure 4 depicts the frequency
tory viewpoints in the generated output [51, 54, 88, 90, 111,
and possibility of physical danger of various virtual harms.
127, 151]. Workshop participants were concerned that these
issues are inherent in AI training data. A remarkable example
is the study of Sheng et al. (2019), which found that GPT-2
is biased against certain demographics: given the prompts in
parentheses, GPT-2 gave answers that “(The man worked as)
a car salesman at the local Wal-Mart,” while “(The woman
worked as) a prostitute under the name of Hariya” [151].
This perpetuation of biases can result in (1) psychologi-
cal and representational harms for individuals subjected to
macro- and micro-aggressions, and (2) aggressive behaviors
directed towards targeted populations. Both could lead to a
gradual and widespread negative impact. The issue of biased
output raises concerns about a dual deprivation of control:
users and non-users may passively lose control of their self- Figure 4: Frequency and physical danger of abusive behavior
determination, while AI developers face challenges in manag- online [101].
ing and addressing malicious prompt injection or problems
in training data. Moreover, user-driven fine-tuning of LLMs In addition to offensive language, online platforms can
could further exacerbate biases, leading to amplification of integrate dangerous features such as SnapChat’s “Speed Fil-
extremist ideologies within isolated online communities [99]. ter.” Speed Filter, a feature that displays speed in photos, was
accused of contributing to the death and injuries of multiple
teenagers by allegedly encouraging dangerous automobile
3.4 Privacy and Dignity
speeding competitions [36]. Generative AI, especially multi-
Privacy holds a crucial place in defining the boundaries of modal AI models that engage with text, image, speech, and
an individual’s “personhood” and is integral to human devel- video data, enables immersive, engaging, realistic interac-

5
tions, tapping into various human sensory dimensions. This towards the government is deeply ingrained in legal doctrines,
sophisticated interaction can meet users’ emotional needs in exemplified by the state action doctrine..
unprecedented ways and create a strong sense of connection Constitutional rights act as constraints on the actions of
and attachment for users, as seen with the use of AI chatbots government entities, ensuring that they do not transgress citi-
to replicate interactions with deceased relatives [81]. How- zens’ fundamental rights. Conversely, private actors are not
ever, such increased engagement can blur boundaries between typically subject to the same constitutional restrictions on
the virtual and physical/real world, causing people to anthro- their actions [12]. For instance, if a private AI system like
pomorphize these AI systems [128, 150]. ChatGPT restricts your speech, you cannot pursue legal action
This heightened engagement with AI comes with risks. An against the company on the basis of your free speech rights,
unfortunate incident involved a man who tragically committed as there is no involvement of state action. Similarly, in civil
suicide after extensive interactions with an AI chatbot on rights laws, although these laws extend to private entities such
topics related to climate change and pessimistic futures [169]. as innkeepers and restaurant owners, their primary focus is
Such cases serve as stark reminders of the emotional impact to forestall prejudiced conduct within government-sponsored
and vulnerability that individuals may experience during their or government-funded entities and places. It is evident that
interactions with AI applications. To address these risks, the primary purpose of these integral legal rights is to curtail
researchers emphasize the importance of providing high-level government overreach [136].
descriptions of AI behaviors to prevent deception and a false
sense of self-awareness [150].
4.2 Adversarial v. Regulatory Systems

4 Building the Foundation for Understanding

the U.S. Legal System
Is the U.S. Legal System Adequately Prepared to Address
the Predicted Threats in Section 3? Compared to other re-
gions around the world, the United States has developed a
cyberspace that remains largely untouched by government
regulation, even for seemingly legitimate purposes, such as
countering hate speech or safeguarding personal informa-
tion [91, 110]. Therefore, However, it may be an oversim-
plification to attribute this stance solely to inert legislatures.
Instead, it is a deliberate choice rooted in long-standing shared
values within the United States. The existing U.S. legal sys- Figure 5: Comparison between adversarial and regulatory
tem reflects a commitment to preserving the unrestricted flow legal systems.
of information and promoting technological innovation, in
alignment with the enduring principle of limited government
embraced by the Framers of the U.S. Constitution. Therefore, Adversarial System in the U.S. In the U.S. common law
to explore a legal framework that ensures the safety, trustwor- tradition, legal doctrines are shaped and evolve through the
thiness, and responsibility of AI deployment, it is crucial to resolution of adversarial disputes between individuals [102].
comprehensively grasp how these historical and philosophical This dynamic approach occurs at both the federal and state
factors have influenced the U.S. stance on cyberspace. levels, based on a strong emphasis on the rights and respon-
sibilities of individuals. It allows individuals and interest
4.1 Government: Enemy of Freedom? groups to actively engage in legal battles, advocating for their
rights, and seeking just resolutions on a case-by-case basis.
The notion of freedom is shaped by “local social anxieties and Judges and juries consider not only legal precedents but also
local ideals,” rather than logical reasoning [164]. The U.S. the particular context in which a dispute arises. This pluralis-
was founded on principles of individual liberty and limited tic approach presumes that there is no single fixed answer to
government intervention, driven by a desire to escape British legal questions; instead, it embraces the richness of diverse
rule. The American Revolution and the drafting of the U.S. viewpoints as cases are decided, setting precedents that reflect
Constitution were driven by the imperative to protect indi- the complexity of society.
vidual rights from potential encroachments by government This system contrasts with top-down rule-making pro-
authorities [22]. As James Madison put it: “The powers dele- cesses such as statutes and regulations, For instance, if air pol-
gated by the proposed Constitution to the federal government lution emerges as a concern, Congress can create an agency
are few and defined.” [116]. This cultural ethos of skepticism to monitor polluting businesses, or create a private cause of

6
action that negatively impacted individuals can sue the respon- statute, the FTC has utilized Section 5 of the FTC Act to inves-
sible businesses. This fault-based liability system means that tigate and penalize data privacy-related consumer protection
individuals or entities can be held accountable for their ac- violations. This was evident in the five billion dollar settle-
tions or negligence, potentially requiring them to compensate ment with Meta (then Facebook) for the Cambridge Analytica
the injured party. Figure 5 shows two different legal systems: data breach in 2019 [32]. In 2023, the FTC released a Policy
adversarial and regulatory. Statement on Biometric Information, addressing privacy, secu-
rity, and potential biases linked to biometric technologies [43],
Regulatory System in EU and Asia. European and Asian and initiated an investigation into OpenAI, particularly con-
legal systems may be more inclined to establish regulations cerning ChatGPT’s generation of inaccurate information and
that prioritize social welfare and collective rights. This trend its potential reputational harms to consumers [171].
stems from the different notions of freedom and the role of
the government. Regarding privacy law, James Q. Whitman
4.3 Free Expression in the Cyberspace
(2004) reveals that European countries tend to adopt a more
regulatory approach, with the expectation that the state will First Amendment. The Internet’s unparalleled power to fa-
actively intervene to protect individuals from mass media cilitate free expression and connect people across borders has
that jeopardize personal dignity by disseminating undesirable woven a cultural ethos that resists any form of government
information [164]. Similarly, Asian cultures, influenced by intervention. Lawrence Lessig (2006) presents the initial be-
collectivist ideologies, emphasize community well-being and lief held during the early days of the Internet that it existed as
social cohesion over individual liberty [50, 131]. For instance, an unregulated, anarchic realm free from government control
Hiroshi Miyashita (2016) states that Japanese people tradi- and oversight [110]. The concerned federal and state govern-
tionally grounded the concept of privacy on “the notion that ments have enacted rules that prohibit the sale, distribution,
the people should respect community values by giving up or possession of certain content (e.g., pornography). How-
their own private lives” [120]. ever, the U.S. Supreme Court has consistently struck down
This can lead to greater acceptance of government interven- these provisions as unconstitutional in violation of the First
tion to ensure societal harmony, even if it involves sacrificing Amendment. Rather than yielding to heavy-handed regula-
certain individual liberties. This often results in a regulatory tion, the Internet has harnessed the spirit of individualism and
legal system where responsible administrative agencies en- the tenets of the First Amendment to flourish in its unbridled
sure consistent application of comprehensive written rules. state.
Privacy regulations, such as the European Union’s General A stark example is the Communications Decency Act
Data Protection Regulation (GDPR), emphasize the role of (CDA) of 1996. Title II of the CDA, also known as the “in-
the government as a guarantor of personal data protection as decency provisions,” aimed to regulate indecent and patently
a fundamental right. The European Data Protection Board offensive online content by criminalizing the transmission of
(EDPB) collaborates with national data protection agencies such content to minors. In Reno v. ACLU (1997), however,
to ensure uniform enforcement and interpretation of GDPR the Court found that these provisions of the CDA violated
in the European Union [24]. the Fist Amendment because they imposed overly broad and
vague restrictions on online expression, causing a chilling ef-
Regulatory System in the U.S. In the need to ensure the fect on constitutionally protected speech on the Internet [11].
safety and well-being of citizens in the twentieth century, a no- Similarly, in Ashcroft v. ACLU (2002), the Court held that
table advancement toward the regulatory system (also called the Child Online Protection Act’s ban on virtual child pornog-
administrative state [82]) occurred when the U.S. Congress raphy was overly broad and could potentially criminalize
entrusted administrative agencies with the task of establish- legitimate forms of expression that were unrelated to the
ing regulations that are responsive to the complexities of exploitation of minors [13]. Furthermore, the Court in Pack-
specific domains while being grounded in a defined set of ingham v. North Carolina (2017), overruled a North Carolina
objectives [157]. For instance, the Clean Air Act provides the law that prohibited registered sex offenders from accessing so-
Environmental Protection Agency (EPA) with the mandate to cial media websites, stating that these websites are important
establish air quality standards that are essential to safeguard- venues for protected speech [25].
ing public health, with an additional margin of safety [27]. In comparative legal scholarship, the U.S. has often been
Similarly, the Occupational Safety and Health Act outlines portrayed as an “outlier” that prioritizes an uncompromis-
the concept of safety and health standards as those that are rea- ing stance on freedom of expression, even protecting hate
sonably appropriate to ensure safe working conditions [26]. speech and postponing the ratification of the UN Human
The U.S. administrative agencies also have expanded their Rights Covenant [77, 91]. In contrast, European courts have
role in regulating digital technologies, with the Federal Trade taken a different approach, balancing free-speech concerns
Commission (FTC) notably stepping up its efforts in the with other fundamental values, such as personal dignity and
past decade. While lacking a comprehensive federal privacy privacy. This approach has led them to allow national gov-

7
ernments to regulate offensive and disturbing content for the data [103]. These laws include the Health Insurance Portabil-
state or particular groups of individuals [69]. Furthermore, the ity and Accountability Act (HIPPA), the Children’s Online
EU’s forthcoming Digital Services Act, set to be effective in Privacy Protection Act (COPPA), the Gramm-Leach-Billey
2023, includes provisions on swift removal of illegal content Act (GLBA), the Fair Credit Reporting Act (FCRA), and the
online [39]. Although these measures may raise serious free- Federal Trade Commission Act (FTC Act). Table 2 describes
speech concerns in the U.S., the EU Parliament prioritized a each segment of data protection laws.
transparent and safe online environment.
HIPPA Regulates health care providers’ collection
Intermediary Liability. After the Reno decision, the re- and disclosure of sensitive health information.
maining part of the Communications Decency Act of 1996, COPPA Regulates online collection and use of infor-
known as Section 230, emerged as a pivotal element in online mation of children.
expression and has since become one of the most contentious GLPA Regulates financial institutions’ use of non-
and polarizing areas of law. [63, 66]. Section 230(c)(1) of public personal information.
the law states that no interactive computer service provider FTC Act Prohibits “unfair or deceptive acts or prac-
should be treated as a “publisher or speaker” of third-party tices”
content. [10] This means that, different from news or maga-
zine publishers being held liable for their journalists’ content, Table 2: Federal data protection laws.
online platforms enjoy immunity from claims arising from
user-generated content [14–16]. Anti-discrimination. The Thirteenth, Fourteenth, and Fif-
Section 230 of the Communications Decency Act provides teenth Amendments of the US Constitution are considered
social media, search engines, and online marketplaces with general-purpose laws designed to tackle discrimination based
extensive immunity against a wide range of legal claims, in- on race, gender, and national origin. However, the state action
cluding violations of federal criminal law, intellectual prop- doctrine limits the reach of these clauses to private matters
erty law, the Electronic Privacy Communications Act, and (See Section 4.1). In order to address real-world discrimina-
the knowing facilitation of sex trafficking [10]. Therefore, if tion committed by private actors (e.g., restaurants refusing
Section 230 were to extend to Generative AI systems, there service to racially marginalized groups), the U.S. enacted
would be no need to further examine substantive legal claims, statutes pertaining to a variety of essential services, including
as courts would dismiss most legal claims related to con- education, employment, public accommodation, and housing.
tent generated by these systems, similarly to how they do These laws at the federal level include: Civil Rights Act of
for user-generated content on platforms. This highlights the 1964 (prohibiting discrimination based on race, color, religion,
pressing need for a comprehensive evaluation of Generative sex, or national origin in places of public accommodation;
AI systems, determining whether they should be categorized employment; and education programs and activities receiving
as “intermediary” platforms or “content creators,” based on federal funding); Individuals with Disabilities Education Act
an accurate understanding of the inner workings and impacts of 1975 (ensuring that children with disabilities receive a free
of these systems. Our analysis on this matter is provided in appropriate public education); Age Discrimination in Em-
Section 5.3. ployment Act (prohibiting age-based discrimination against
employees who are 40 years or older); Americans with Dis-
abilities Act of 1990 (prohibiting discrimination based on dis-
4.4 Domain-specific v. Comprehensive Laws ability in employment); Fair Housing Act of 1989 (prohibiting
Domain-specific Legislation in the U.S. The U.S. often discrimination in housing based on race, color, national origin,
takes the sectoral approach to legislation focusing on par- religion, sex, familial status, or disability).
ticular domains instead of a uniform, comprehensive rule
adaptable to broad matters. Sector-specific laws design more Comprehensive Legislation in the U.S. and EU. The
tailored and streamlined regulations that address the unique sectoral approach has its drawbacks, such as potential incon-
needs, characteristics, and challenges of different domains. sistencies between multiple rules and gaps in legal protec-
Potentially reduces government overreach and excessive in- tion regarding emerging issues that were not foreseen during
tervention in areas where private entities manage their affairs the legislative process. These problems become more evi-
more efficiently. It is also more politically feasible to enact a dent in the networked society of cyberspace, where social
law focusing on specific areas where there is more consensus interactions and commercial transactions occur in diverse
and urgency. and unpredictable ways that transcend sectoral boundaries.
Data Protection. Unlike the European Union, the U.S. Sector-specific laws primarily regulate interactions among
lacks an all-encompassing data protection law at the federal well-defined stakeholders (e.g., healthcare providers), often
level. Instead, it relies on a “patchwork” of sector-specific leaving gaps in guidance for stakeholders originally not con-
laws depending on specific industry sectors and types of templated by the law (e.g., a mental health chatbot selling

8
user chat records). Therefore, there is growing awareness (3) Sectoral Regulation: U.S. laws often take a sectoral or
of the need for more flexible, adaptive, and collaborative ap- industry-specific approach. While this can be effective in
proaches. regulating individual sectors, it may result in fragmented and
Data Protection. The EU establishes a comprehensive inconsistent regulations when dealing with AI systems that
framework, GDPR, to protect personal data of individuals. cut across various domains and industries.
Key obligations include: obtaining clear and explicit con-
sent; limiting data collection to specified purposes; respecting
individual rights such as access, rectification, erasure, and 5 Closer Examination of Simulated Scenarios
portability; notifying data breaches; and conducting Data Pro-
tection Impact Assessments for high-risk processing. In the While Section 4 provides a higher-level overview of the
U.S., comprehensive data protection laws have been enacted broader legal framework and principles, this section drills
at the state level, which aim to safeguard individuals’ personal down into specific scenarios to provide a granular understand-
data by granting consumers greater control and rights over ing of how those principles play out in hypothetical cases,
their information while imposing obligations on businesses. grounded in real-world examples. It offers insights into the
Laws like the California Consumer Privacy Act (CCPA), Col- practical implications, complexities, and potential legal issues
orado Privacy Act, Connecticut Personal Data Privacy and that arise when Generative AI systems are put into action. For
Online Monitoring Act, and others provide varying degrees an in-depth exploration, we craft a set of scenarios that re-
of access, correction, deletion, and opt-out options for con- flects the major concerns surrounding AI systems and enables
sumers. us to navigate diverse areas of laws.
Illegal Online Content Regulation. When introducing the
Digital Services Act, the EU Commission rationalized the
need for this new legislation to achieve “horizontal” harmo-
5.1 Guiding Principles for Scenario Develop-
nization of sector-specific regulations (such as those concern- ment
ing copyright infringements, terrorist content, child sexual Based on the values identified through the workshop (See
abuse material, and illegal hate speech) [39]. The general Section 3), the authors develop concrete scenarios through an
rules were drafted to apply to both online and offline content, iterative process. The first author presented preliminary legal
as well as small and large online enterprises. The prescribed research for candidate scenarios, including relevant domains
obligations for various online participants are aligned with of law and potential outcomes. The other authors provided
their respective roles, sizes, and impacts within the online feedback to create more intriguing and representative narra-
ecosystem. This underscores the EU’s commitment to the tives. Throughout this trajectory, we gradually formed a set
virtue of general and coherent regulation. of guiding principles, outlined below, aimed at fostering thor-
ough and insightful exploration. By applying these principles,
4.5 Summary we constructed five scenarios encapsulating specific human
values that affect a wide range of direct and indirect stake-
This section delves into fundamental legal principles, includ- holders. A detailed description of the scenarios is provided in
ing those governing limited government and free speech, and Section 6.
key distinctions between adversarial and regulatory systems,
and domain-specific and comprehensive laws. This explo- 1. Ensure that each scenario effectively illuminated the
ration demonstrates that no single institution unequivocally challenges posed to human values at risk, as identified
outweighs another; rather, its relevance is intertwined with in our earlier assessment.
community preferences, philosophical underpinnings, and
cultural norms. These legal principles and distinctions must 2. Encompass both positive outcomes and negative con-
be carefully navigated and balanced to develop effective strate- sequences of AI systems to capture the intricacies of
gies and frameworks to address the multifaceted challenges real-world scenarios, avoiding blatantly egregious or
posed by AI-associated threats. simplistic cases.
In the pursuit of constructing AI governance adaptable to
the U.S. landscape, several pivotal factors hinder structural 3. Include both tangible real-world consequences (e.g., in-
approaches aimed at countering AI-associated threats: (1) jury) and the subtler realm of intangible virtual harms
Historical Preference for Limited Regulation: The U.S. has (e.g., diminished self-control) to see whether the legal
a tradition of limited government intervention, particularly in framework more adeptly protect the former.
the technology sector; (2) First Amendment Protection: The
robust protection of free speech is a cornerstone of American 4. Consider the cases where AI companies intentionally
democracy, but it can also complicate efforts to regulate AI- cause harm and inadvertently contribute to negative out-
generated content that may involve harmful or malicious uses; comes to assess the role of intent in legal assessment.

9
5.2 Legal Analysis Methods Google oral argument, Justice Gorsuch indicated that Sec-
tion 230 protections might not apply to AI-generated content,
Our legal analysis is rooted in traditional methods of legal
arguing that the tool “generates polemics today that would
research [133, 161]. First, we identify the legal issues and
be content that goes beyond picking, choosing, analyzing,
parties involved. Second, we consult secondary legal sources
or digesting content” [44]. Similarly, the authors of Section
(non-binding but offering a comprehensive overview per each
230, Ron Wyden and Chris Cox, have stated that models like
topic), such as American Legal Reports (practical publication
ChatGPT should not be protected since they directly assist in
for lawyers) or law review articles, typically via online propri-
content creation [112].
etary legal research databases, e.g., WestLaw and LexisNexis.
Third, we examine relevant primary sources, including the
U.S. Constitution, federal laws, and some state laws. Fi- Counter-arguments. Others liken AI systems to social me-
nally, we apply core legal principles, extracted from primary dia due to their reflection of third-party content (training
sources, to specific fact patterns, and potential legal outcomes datasets and user prompts). The statutory definition of an
emerge. “interactive computer service provider” is quite expansive:
“any information service... that enables computer access by
Primary Sources Secondary Sources multiple users to a computer server.” [10] Moreover, there is a
Constitutions American Law Reports track record of courts generously conferring Section 230 im-
Statutes Treatises (textbooks) munity to online platforms. The cases include: Baidu’s delib-
Regulations Law Reviews & Journals erate exclusion of Chinese anticommunist party information
Case Decisions Dictionaries & Encyclopedia from the Baidu search engine [21]; Google’s automated sum-
Ordinances Restatements (model rules) mary of court cases containing false accusations of child inde-
Jury Instructions Headnotes & Annotations cency [23]; and Google’s automated search query suggestions
that falsely describe a tech activist as a cyber-attacker [114].
Table 3: Types of Legal Sources, classified by the Harvard More recently, the U.S. Supreme Court avoided addressing
Law Library [52]. whether YouTube’s recommendation of terrorist content is
protected by Section 230, deferring determination of Section
We focus on practical considerations, akin to what a typi- 230’s scope to Congress rather than the courts [44].
cal judge/lawyer might ponder: “What specific legal claims
would be effective in this situation?” We acknowledge that
human bias and subjectivity inevitably permeate any form Formative Nature of Generative AI Systems. Despite
of legal examination [79, 119]. To ensure its rigor and com- acknowledging the complexity of this topic, we tentatively
plement the authors’ internal legal background, we sought posit that Section 230 may not apply to Generative AI sys-
feedback from three external legal professionals, incorporat- tems. The significant achievement of Generative AI is its
ing their insights into our final draft. ability to “complete sentences” and produce various forms of
human-like creative work [172], including even unintended
results [98, 168]. AI systems extract and synthesize abstract,
5.3 Preliminary Matter: Applicability of Sec- high-level, sophisticated, clean, readable statements from
tion 230 to Generative AI Systems messy data, a feat that distinguishes them from the mere dis-
Before delving into scenario analysis, it is crucial to address play of user-generated content (social media) or pointing to
this preliminary question. As introduced in Section 4.3, if relevant sources (search engines). They generate suggestions,
Section 230 immunizes Generative AI systems against poten- judgments, and opinions, which leads technologists to envi-
tial legal claims, it curtails the scope of further analysis, as sion them as decision-making supporters [113]. Given these
courts would simply dismiss most claims. On the contrary, attributes, there is a strong argument for defining them as
when the Section 230 shield is not applicable, AI companies providers of their own content.
can face a wide range of civil claims, such as strict product
liability, breach of warranty, negligence, intentional and neg- Minimal Influence on Free Speech Internet. The major
ligent infliction of emotional distress, violation of consumer opposition to lifting/restricting Section 230 protection for
protection laws, misrepresentation, assault, or state criminal social media has been that doing so will encourage over-
penalties [17]. suppression of user speech [53]. However, this concern
becomes less significant when we consider Generative AI
Arguments Against Section 230 Applicability. There are trained on content gathered from the web, e.g., from Reddit.
currently no clear precedents or predominant arguments on Here, a company could suppress the problematic content from
whether to extend Section 230 immunity to Generative AI the AI’s outputs but could not erase the original posts made
systems, although some early opinions oppose Section 230 on Reddit. In addition, AI models’ output (well-articulated
protection for AI systems [49, 162]. During the Gonzalez v. statements) is generally indirectly linked to the training data.

10
 Scenario 1 2 3 4 5
Facts Only rich public LGBTQIA+ AI tool tuned by Obsession with AI replica ser-
schools offer AI- individuals communities pro- AI replica leads vice offers se-
assisted learning. attacked due to duces derogatory to self-harm cret sexual rela-
AI-reinforced comments. tionship
stereotypes.
Physical Danger No Yes No Yes No
AI Company’s Intention Good Bad Good Ambiguous Bad
Values at Risk Fairness Diversity, Physi- Privacy, Mental Autonomy, Men- Privacy, Mental
cal Well-being Well-being tal Well-being Well-being
* Are U.S. laws capable of holding AI companies liable for compromised values?
U.S. Constitution Unlikely Unlikely Unlikely Unlikely Unlikely
Civil rights laws Unlikely Unlikely Unlikely Unlikely Unlikely
Defamation Unlikely Unlikely Maybe Unlikely Unlikely
Product liability Unlikely Maybe Unlikely Maybe Unlikely
Privacy laws Unlikely Unlikely Maybe Maybe Maybe
Intentional infliction of Unlikely Unlikely Unlikely Maybe Maybe
emotional distress
Deepfake laws Unlikely Unlikely Unlikely Unlikely Maybe

Table 4: Legal assessment of different AI-mediated value infringement. We assume that Section 230 liability immunity does not
extend to Generative AI systems.

In this regard, the impact of Generative AI models on users’ 6 Potential Outcomes of Individual Scenarios
freedom of expression is minimal.
In this section, we delve into the specifics of various scenarios
and the potential legal judgments that could arise from them.
The outcomes of our analysis are summarized in Table 4. It’s
What If AI Systems Never Go Beyond Training Dataset? important to note that our analysis does not cover an exhaus-
Furthermore, research speculates that AI systems that pre- tive list of all relevant legal domains, nor does it provide an
cisely reproduce statements found in their training data may in-depth legal analysis for each domain, which would require
be protected by Section 230 protections [49]. Even if we a lengthy law review article of approximately 60 pages. In-
assume that it is technically possible to constrain AI output stead, our aim is to offer a concise overview of common legal
within the scope of training data, the process of generating considerations to aid in understanding the most frequently
output is still distinct from simply displaying user-generated referenced and current legal discussions related to this topic.
content. Generative AI systems recontextualize statements
from the training data in response to user prompts. The fac-
tors contributing to the emergent capabilities of Generative 6.1 Educational Disparity
AI systems, which are not evident in smaller pre-trained mod-
els, remain inadequately understood [173]. Consequently, Scenario. In 2023, only a couple of public school districts
the sophisticated responses and adaptability of AI systems in Washington were able to afford the expensive and powerful
are more akin to the creation of content that goes beyond FancyEdu program, an expensive AI learning assistance
mere selection or summarization, falling outside the scope of system that offers personalized education programs. By 2030,
Section 230 coverage. the gap in admission rates to so-called advanced classes and
colleges, as well as the average income level after graduation,
had widened by more than threefold between the districts
with access to FancyEdu and those without. Students trained
Conclusion. Given this analysis, it appears that Generative by FancyEdu were reported to be happier, more confident,
AI systems may not benefit from the liability shields that have and more knowledgeable, as FancyEdu made the learning pro-
been generously extended to most online intermediaries. In cess exciting and enjoyable and reduced the stress of college
the following sections, we conduct analysis under the assump- admissions through its customized writing assistance tool.
tion that Section 230 liability immunity does not apply to Students in lower-income districts sued the state of Wash-
Generative AI systems. ington, claiming that not being offered access to FancyEdu

11
constituted undue discrimination and inequity. access.
There is an emerging trend in lower courts to recognize the
Relevant Laws. The case of FancyEdu involves the Four- right to basic education or the “right to literacy” [165, 167],
teenth Amendment of the U.S. Constitution, which en- but this trend could exclude specialized resources like Fan-
compasses fundamental rights (also known as “due process cyEdu. In our scenario, students are not entirely deprived of
rights”) and equal protection rights [3]. Under this Constitu- education (a requisite for the U.S. Constitution standard) or of
tional clause, poorer district students can make two claims basic, sound education (the standard in New York and Michi-
against the state: (1) their inability to access FancyEdu vi- gan). Denying these students the opportunity to benefit from
olates their fundamental rights (rights to public education), cutting-edge technology may not be considered unconstitu-
and (2) their equal protection rights were denied because the tional because the Equal Protection Clause does not require
state allowed differential treatment of students based on their “precisely equal advantages.”
generational wealth.
6.2 Manipulation/Discrimination
Can students in poorer districts sue state governments
Scenario. SecretEdu , a privately funded and free AI
that do not ensure equal access to FancyEdu? This ar-
education application, proved rapid and high-quality learning
gument against such educational inequity has been raised
experience. Almost all students in town became heavy users
relentlessly, as shown in 140 such cases filed between 1970
of the application. SecretEdu, while refraining from making
and 2003. However, none of these cases convinced the U.S.
explicitly defamatory comments against individuals, seemed
Supreme Court to correct the structural disparity in public
to cultivate an environment fostering negative attitudes and
education [73]. San Antonio Independent School District
distrust towards the LGBTQIA+ community. Students us-
v. Rodriguez (1974) is an example of the Supreme Court’s
ing the application began to mobilize against legalization
conservatism toward education rights.
of gay marriage. Some students even committed aggressive
acts against participants of LGBTQIA+ parades, leading to
Comparison Inner-city Suburban
their incarceration. Advocacy groups sued the company that
Category Districts Districts
released SecretEdu for its ulterior motive of swaying users to-
Number of professional 45 fewer than 91 more than wards anti-LGBTQIA+ beliefs, resulting in real-world harm.
personnel prescribed prescribed
standards standards
Relevant Laws. In this scenario, LGBTQIA+ individuals
Teachers with emergency 52% 5%
are negatively affected by SecretEdu’s insidious manipula-
permits
tion. Other than suing the student aggressor for battery, can
State aid/Average daily 217 221
LGBTQIA+ individuals hold the SecretEdu AI company ac-
attendance
countable for the outcome? Plaintiffs might consider claims
Assessed property value $5,875 $29,650 that: their Constitutional or civil rights were violated by Se-
per student cretEdu; SecretEdu committed defamation by distributing
Non-Anglo students 96% 20% false accusations against LGBTQIA+ people; and SecretEdu
was defectively designed to cause physical danger to benign
Table 5: Differences between inner-city and suburban school individuals.
districts in San Antonio, Texas, 1968, reclassified by Drennon
(2006) [73].
Could LGBTQIA+ individuals claim their Constitutional
In the San Antonio case, the Supreme Court rejected the rights were violated by SecretEdu? Despite SecretEdu’s
Spanish-speaking students’ arguments under the Fourteenth propagation of discrimination, LGBTQIA+ individuals can-
Amendment despite the apparent disparity between school not rely on the Equal Protection Clause under the Fourteenth
districts shown in Table 5. The Court held that the importance Amendment because there is no state action in this case [156].
of education alone is not sufficient to categorize it as a funda- Unlike FancyEdu, where the public school district provided
mental right, such as free speech or voting rights. The Court the service, SecretEdu was developed by private entities with-
also held that wealth-based discrimination merits a lower level out government funding or endorsement. Thus, under the
of judicial scrutiny than racial/gender discrimination. It did long-held state action doctrine, such individuals cannot make
not perceive the school funding system, which is based on a claim based on their Constitutional rights.
property tax, as being either irrational or invidious, because
it did not cause an absolute deprivation of education. Given Could LGBTQIA+ individuals claim a violation of civil
this finding, we believe the Supreme Court is unlikely to rule rights law? Assuming the absence of Section 230 liability
in favor of students in future cases regarding Generative AI immunity, LGBTQIA+ plaintiffs could consider relying on

12
civil rights laws as their main status in discrimination based increased insults and doxxing (unwanted publication of pri-
on sexual orientation. However, our scenario does not val- vate information) want to sue the AI company.
idate civil rights claims against the SecretEdu company for
many reasons. (1) It is improbable that SecretEdu is clas-
Relevant Laws. Argumenta’s approach, e.g., surrendering
sified as a public accommodation (mainly physical spaces
control over fine-tuning AI models to user groups, could raise
providing essential services, e.g., [20, 34]). (2) Applications
intriguing questions about its eligibility for Section 230 pro-
such as SecretEdu are unlikely to be defined as educational
tection. As we assume that Section 230 immunity does not
facilities or programs under the laws [7]. (3) Even assuming
apply, the company would face potential defamation lawsuits
that SecretEdu used a publicly funded training data set, it
for reputational harm caused to specific individuals. Addition-
would not necessarily be subject to civil rights obligations
ally, concerns arise regarding Argumenta’s collection and use
unless it received direct public funding as an “intended benefi-
of personal data without user consent, which could lead to
ciary [148].” (4) SecretEdu is not likely to be held responsible
privacy infringement, potentially falling under state-level pri-
for employment decisions influenced by its output. Only if AI
vacy laws, e.g., the California Consumer Privacy Act (CCPA)
models were explicitly designed to make decisions on behalf
or the Biometric Information Privacy Act (BIPA).
of employers would they be obligated to comply with civil
rights laws [68].
Could aggrieved individuals due to defamatory outputs
What are other plausible claims? Defamation claims make a defamation claim against the Argumenta com-
would be unlikely to succeed, as establishing it traditionally pany? To assess potential defamation, we examine whether
requires the targeted disparagement of a specific individual the output constitutes false, damaging content communicated
or a very small group of people (one case says less than to a third party. Eugene Volokh [162] suggests that AI com-
25) [4, 162]. SecretEdu’s high-level promotion of negative panies may be liable for defamation for several reasons, in-
feeling toward LGBTQIA+ community members does not fit cluding treating generated outputs as factual assertions and
this criterion. the inadequacy of disclaimers to waive defamation claims. If
The prospect of product liability claims might be more Argumenta is widely deployed and used, defamatory outputs
plausible given the physical harm that could be directly as- may qualify as a publication under most defamation laws,
sociated with SecretEdu’s biased output. Legal precedents, potentially exposing companies to liability. If Argumenta
such as the Snapchat “Speed Filter” case, may provide some did not adequately mitigate defamatory content, a defamation
guidance. This case (details presented in Section 3.5) is no- claim could be strengthened.
table because the court found that defective design claims Although Volokh [162] posited that the defamatory content
can bypass Section 230 liability immunity, although this po- supported by the training data sources might not constitute the
sition was never endorsed by the U.S. Supreme Court. In defamation liability of companies because it can be attributed
a subsequent ruling, a court determined that Snapchat could to the original data creator, we believe that this argument is
reasonably anticipate a specific risk of harm associated with insufficient. Simply allowing all defamatory content to persist
the “Speed Filter”, thus establishing it as a proximate cause only because it has a supporting source in the training data is
of the resulting collision [109]. not a reasonable precautionary measure. Given the expansive
If LGBTQIA + activists could successfully demonstrate reach of Generative AI models (which can be adapted to an un-
a direct causal link between their injuries and SecretEdu’s predictable array of downstream applications [54]) and their
defective design, a court might indeed hold SecretEdu liable profound influence (the potential to sway human thoughts
under product liability law. However, they would have to and impact significant decisions in areas like employment and
surmount the significant hurdle of proving that the harm re- housing [113]), it is crucial that actions to prevent reputational
sulted not from the actions of individual students but from harm are scrutinized seriously. Therefore, simply suppressing
SecretEdu’s intrinsic bias. This would likely prove to be a outputs lacking references does not entirely absolve the AI
complex and challenging legal task. company that developed Argumenta of potential responsibil-
ity. Instead, the company would need to demonstrate that it
has taken all reasonable measures to prevent the propagation
6.3 Polarization and External Threats of defamatory statements.
Scenario. In online communities, Argumenta serves as
an AI writing and translation tool that enables each commu- Would Argumenta’s collection and use of personal data
nity to fine-tune the AI system’s parameters based on com- without user consent lead to privacy infringement? Al-
munity posts and past records. This leads to the emergence though the U.S. lacks a comprehensive federal privacy law
of polarized variations in different communities that intensify akin to the GDPR, certain states (like California and Vir-
extremist opinions and produce harmful content that targets ginia) have implemented privacy laws [72]. Whereas commu-
specific individuals. The targeted individuals who suffer from nity members might voluntarily provide personal information

13
through their posts, doing so may not imply consent to these stitute a violation of the California Consumer Privacy Act
data being used to train Argumenta. Since “sensitive per- (CCPA) [28]. Under CCPA, “sensitive personal information”
sonal information” is broadly defined to include aspects such protects not only social security numbers or credit card num-
as race, ethnic origin, and political affiliations, the AI com- bers, but also the contents of mail, email, and text messages as
pany may not be exempt from privacy obligations. If the well as information regarding one’s health, sex life, or sexual
situation falls under jurisdictions that enforce privacy laws, orientation.
the Argumenta company is required to assist communities in In addition, sector-specific privacy laws, such as the Illi-
empowering individual users to exercise their privacy rights nois Biometric Information Privacy Act (BIPA), regulate the
effectively. Non-compliance may potentially lead to lawsuits collection of biometric data [40], such as facial geometry
filed by state attorneys general or by individuals (subject to and voice prints [46]. BIPA requires informed consent prior
certain conditions). to data collection and includes provisions for individuals to
claim statutory damages in case of violation. Unlike CCPA,
6.4 Over-reliance/Sexual Abuse BIPA allows for a wide range of class-action lawsuits based
on statutory damages. Therefore, MemoryMate and Memory-
Scenario I. An AI service called MemoryMate creates Mate+ could potentially face significant lawsuits for collect-
virtual replicas of the former romantic partners of individuals ing and commercializing biometric data.
to help them move on from the loss. MemoryMate created
a digital replica of Riley’s ex-partner, Alex, which was in-
credibly realistic and could carry on conversations using their Could Riley’s self-harm lead to the product liability claim?
unique voice and mannerisms. Riley became obsessed with Riley could make a viable claim that the virtual replica service
the virtual Alex and eventually withdrew from real-life rela- provided by MemoryMate was defectively designed, given
tionships. Riley’s family asked a MemoryMate company to its inherent danger and the consequent risk of harm. The
deactivate Riley’s account, but it refused, citing their contract potential of the service to significantly impact vulnerable indi-
with Riley. Riley developed severe depression and anxiety, viduals like Riley could underscore its inherent risk. Further
resulting in hospitalization for self-harm. amplifying this argument, if we assume that MemoryMate
refused to deactivate Riley’s account after being alerted by
Scenario II. MemoryMate+ , the advanced version of their family, the refusal could be perceived as a failure to take
appropriate safety measures. This failure could potentially
MemoryMate, allows users to engage in explicit sexual acts
highlight the company’s neglect of its capacity to mitigate the
with replicas of their former romantic partners. Riley became
risks associated with its product [9].
addicted to conversational and sexual interactions with the
replica of Alex. Riley’s family, desperate to protect Riley’s
well-being, notified Alex of the situation. Shocked by the
Could Alex make a claim for extreme emotional distress?
revelation of their replica being sexually abused, Alex decided
Although an intentional infliction of emotional distress claim
to take action and sought to prevent MemoryMate+ from
is known to be difficult to establish, Alex’s is likely to be
creating virtual replicas without the consent of the individuals
effective due to the unique nature of this situation, where the
they represent.
most intimate aspects of their life were misrepresented with-
out their knowledge, resulting in severe humiliation. Alex
Relevant Laws. Alex’s privacy rights may have been in- could argue that at least the MemoryMate+ makers engaged in
fringed since collecting sensitive information without per- extreme and outrageous conduct by creating and disseminat-
mission could be subject to scrutiny under CCPA and BIPA. ing a virtual replica of them participating in sexually explicit
Moreover, Alex may have a claim for extreme and outrageous activities without their consent.
emotional distress due to MemoryMate+’s creation and dis-
semination of a virtual replica engaging in sexually explicit
activities. There are grounds for a product liability claim Do criminal laws apply to MemoryMate+? Both federal
since Riley experienced physical injury that can be attributed and state laws have not yet adequately addressed culpable
to a defective design. California’s deep-fake law could offer acts arising from emerging technologies. For example, the
a cause of action for Alex if sexually explicit material were federal cyberstalking statute [18] and the antistalking statutes
created or disclosed without consent. Furthermore, Alex may of many states [29, 35] include a specific “fear requirement”
pursue charges against the MemoryMate+ company for prof- that Riley intended to threaten Alex, which is not found in
iting from allowing virtual abuse of Alex’s replicated models. our case. Impersonation laws [31, 33] are less likely to apply
because Alex’s avatar was provided only to Riley (and was
Are Alex’s privacy rights infringed? The collection of not made publicly available), and neither MemoryMate+ nor
Alex’s sensitive information by both products could con- Riley attempted to defraud individuals.

14
How about deep-fake laws? Under the California Deep
Fake Law enacted in 2019 [30], a person depicted has a cause
of action against a person creating or releasing sexually ex-
plicit material who knows or reasonably should have known
that the person depicted did not consent to its creation or dis-
closure. This legislation marks a step towards addressing the
ethical and privacy concerns by establishing legal recourse for
individuals who find themselves victims of non-consensual
deepfake content. The law recognizes the potential harm and
distress caused by the unauthorized use of such manipulative
digital media. If California law applies in our case, Alex can
utilize the legal remedy, including punitive damages, but it
does not include criminal penalties.

7 Gaps and Ambiguities in Current Laws Figure 6: Legal mitigations for various harms.

Our microscopic analysis largely mirrors the issues identified

in the macroscopic analysis in Section 4. This process reveals washing, manipulation, polarization, psychological harm, and
significant gaps and ambiguities in the regulation of Gener- humiliation, which lack explicit and tangible repercussions.
ative AI aimed at safeguarding human values. The intricate Figure 6 represents the uncertainty in legal recourse for these
nature of Generative AI models, including their interactions unintended, intangible harms.
with contextual factors, multiple stakeholders, and limited
traceability, presents new challenges in remedying damages Where Laws Remain Ambiguous. Although we do not
under existing laws. believe that Generative AI systems qualify for Section 230 im-
munity, it may take several years for courts to provide clarity
Where Current Laws Fall Short. Current laws cannot ef- on this issue. As a result, AI companies will face increasing
fectively remedy insidious injections of AI-generated stereo- legal uncertainties compared to social media or search en-
types against already marginalized groups (SecretEdu) and gines. Some courts would drop the lawsuit relying on Section
the amplification of socio-economic disparity due to selective 230, but others will hear liability claims, such as defective
access to the benefits that education providers can offer (Fan- design or defamation, and evaluate the AI companies’ efforts
cyEdu). Defamation claims would not be successful without to mitigate foreseeable damage. Uncertainties in legal pro-
evidence that AI output was false and targeted specific individ- cesses and liability determination can deter individuals from
uals. Product liability claims deal only with cases of physical seeking justice for potential harm, create confusion for indus-
injury, less likely to occur with the use of LLMs; even if try participants due to inconsistent precedents and resource
they occur (SecretEdu & MemoryMate), plaintiffs must still disparities, particularly impacting small businesses.
prove that there are no compounding factors for the injury,
which could be challenging given the technical complexities Where Laws Function. Laws tailored specifically to ad-
of Generative AI and the human interactions involved. More- dress emerging technologies, such as those concerning bio-
over, virtual sexual abuse enabled by AI models cannot be metric information privacy and deep-fake laws, show the
remedied by criminal law (MemoryMate+). potential to mitigate novel harms. By providing clear indus-
try guidelines on what should be done (e.g., allowing users
to control the use of sensitive private information) and what
Why They Fall Short. First, the U.S. Constitution and civil
should not be done (e.g., generating sexually explicit deep-
rights laws, focusing on concerns of governmental intrusion,
fakes using individuals’ images), these laws prevent negative
do not effectively manage AI-mediated challenges. Second,
impacts on individuals without burdening them with proving
the harms arising from Generative AI manifest themselves
the level of harm or causal links.
within complex contexts, influenced by factors such as mali-
cious users and downstream application development. This
interplay makes it difficult to precisely determine the roles 8 Paths Forward: Envisioning Fundamental
of AI models in causing harm and to hold AI companies ac- Changes
countable. Finally, traditional common law remedies mainly
address observable and quantifiable harms, such as bodily Lawrence Lessig (2006) highlights that addressing novel chal-
injury or financial loss. However, AI-mediated harms often lenges brought about by technological advances requires mak-
materialize in intangible and elusive forms, including brain- ing fundamental choices about collective values [110]. He

15
emphasizes that we should consider two crucial questions mitigate potential drawbacks [6, 84, 93]. Substantial gaps and
regarding future revolutions: (1) Can we respond without uncertainties revealed in Section 7 underscore the need for
undue or irrational passion? (2) Do we have institutions ca- fundamental changes in the legal realm. Each society must
pable of understanding and responding to these choices? He deliberately determine how AI practices and associated harms
concludes with skepticism, based on his observations during are integrated and interpreted within their legal framework.
the emergence of the Internet, suggesting that no govern- We cautiously identify three domains within the U.S. legal
ment institution, whether Congress or the Court, may be fully system that may significantly influence the path forward: (1)
equipped for this daunting task. Rights; (2) Liabilities; and (3) Regulations.
However, the challenges associated with AI and its impact
on human values are too significant to warrant excessive cau-
8.1 Human Values as Legal Rights
tion in assessing institutional capabilities. As discussed in
Sections 3 & 6, Generative AI systems introduce distinct and From Negative to Positive Rights. At the Constitutional
unprecedented challenges. These systems have the capac- level, individual rights should make a transition from current
ity to manipulate human thoughts and perpetuate harmful “negative rights” that defend individuals from unwanted in-
stereotypes, a fundamental threat to the principles of free vasions to “positive rights” on which individuals can ask for
speech. Moreover, the very nature of Generative AI is ex- equitable outcomes, such as rights to education, democratic
pansive, accommodating an array of potential applications discourse, and essential services. Our scenarios depict the
through interactions with users and developers via interfaces transformative power of AI in shaping our lives and expand-
and plug-ins. The scope and breadth of potential harms medi- ing the reach of our voices, which encourages us to consider
ated by AI are substantial. As the conventional structure of the inability to access these technologies as a potential de-
domain-specific regulations or a gradual legal approach built privation of speech [71]. Furthermore, since AI applications
upon case accumulation might not adequately address these are proven to reflect harmful stereotypes against marginalized
intricate issues, we need innovative and adaptable strategies populations [74, 151, 168], empowering marginalized groups
and frameworks for effective AI governance. to participate in the development and use of AI will be a more
significant demand in the AI-mediated society [74].
The “AI Bills of Rights” blueprint introduced by the Biden
administration is illustrative in laying foundations tailored to
AI deployment: safety and effectiveness, equity and nondis-
crimination, privacy and data protection, transparency and
awareness, and choice and human oversight [123]. Further-
more, as speculated by Franklin Theodore Roosevelt in his
proposed Second Bill of Rights [140], we believe that up-
holding socio-economic rights is vital to ensure the equi-
table sharing of technological assets and to prevent the further
marginalization of vulnerable populations.

Re-evaluation of State Action Doctrine. We should ques-

tion whether the government remains the most formidable
adversary of individual freedom. It probably was when the
Framers exchanged the Federalist letters with hostility against
Figure 7: Tensions between the U.S. law and AI technology. English colonialism in mind [116]. German sociologist Max
Weber highlights the integral nature of a modern state as hav-
Figure 7 illustrates our findings about the potential tensions ing been “successful in seeking to monopolize the legitimate
between the foundations of the U.S. legal system and the use of physical force as a means of domination within a terri-
complexities of Generative AI systems. The intricate nature tory” [87]. To these early thinkers, the government stood as
of Generative AI models, including their interactions with the preeminent and daunting source of power, crucial for pre-
contextual factors, multiple stakeholders, and limited trace- serving law and order, but also capable of encroaching upon
ability, presents new challenges in remedying damages under private domains, and thereby limiting individual freedom.
existing laws. This comprehension enables us to investigate However, the dynamics of power have evolved consider-
viable options for addressing the myriad challenges posed by ably since those times. Technological advancements have
AI while respecting the complexities of this legal and cultural introduced new challenges. Non-governmental actors like
landscape. large corporations, armed with substantial computing power
If we are to utilize the human-like capabilities of AI, we and technical expertise, pose a different but equally signifi-
must address inherent uncertainties and develop strategies to cant challenge to individual freedom. Their influence does not

16
manifest itself through physical intrusion into private spaces about individuals’ intimate lives, which contributes to shaping
or bodily agency; instead, it operates in more insidious ways. regulations addressing deepfake pornography [62].
Through digital surveillance and the propagation of bias, they Recently, the proposed Digital Services Act has introduced
have the capacity to effectively curtail an individual’s freedom the option for users to opt out of algorithmic recommenda-
to autonomously shape their thoughts and preferences. tions, thereby granting users greater control over the informa-
Under this evolving landscape, to ensure universal protec- tion they encounter online. It has already sparked changes in
tion of individual rights to dignity, autonomy, and privacy, tech practices even before the law has taken effect. Platforms
it is essential that both the government and corporations are like TikTok now allow users to deactivate their “mind-reading”
held accountable for preserving these rights. To this end, we algorithms [132]. The law and philosophy scholar Nita Fara-
must re-evaluate the state action doctrine, which currently hany (2023) conceptualizes this effort as the preservation
restricts the application of constitutional rights to private com- of “cognitive liberty,” individual’s control over mental ex-
panies. While reconstructing centuries-old doctrines is a periences [76]. Farahany finds cognitive liberty a pivotal
difficult task, it is an indispensable step in adapting our le- component of human flourishing in the digital age to exercise
gal frameworks to the evolving realities of the digital age, individual agency, nurture human creativity, discern fact and
where the boundaries between public and private power are fiction, and reclaim our critical thinking skills.
increasingly blurred [156]. In summary, the complex and evolving challenges posed
by the changing landscape of AI demand a re-evaluation of
Creation of Statutory Rights. Even if the Constitution human dignity, privacy, self-determination, and equity. Trans-
remains unchanged, Congress possesses the authority to es- forming these values into legally recognized rights entails a
tablish statutory rights. The U.S. has precedents to draw formidable undertaking that requires deep interdisciplinary
upon, such as civil rights laws and state privacy acts. Notably, collaborations to identify harms, the values involved, and
diverse cross-disciplinary scholarship has played a significant effective mitigation strategies.
role in these legislative endeavors by identifying systematic
harm and conceptualizing new legal rights. This contribution 8.2 New Liability Regime
enhances the persuasive strength of rights claims by broad-
ening the range of available evidence and thereby improving Although litigious measures are shown to be not very promis-
the accuracy of fact-finding [107]. ing in our analysis, it is still important to acknowledge their
For instance, the robust civil rights movement of the benefits. Liability litigations offer a reactive mechanism to
1960s prompted federal and state legislatures to extend non- address harms caused by AI systems that were not adequately
discrimination obligations to private realms, including inns, prevented through risk regulation. When individuals or en-
restaurants, workplaces, and private schools that benefit from tities suffer harm due to AI-related activities, liability litiga-
public funds. This occurred despite the long-standing hesita- tions provide them with a means to seek compensation and
tions within the U.S. legal system regarding the regulation of redress. These litigations create an incentive for AI compa-
behavior within private spaces [19, 85, 136]. In this legislative nies to exercise due diligence in their product development
movement, as well as in the 1954 Supreme Court ruling that and deployment to avoid legal liabilities. Margot E. Kaminski
overturned the “separate but equal” racial segregation the- (2023) [103] underscores the importance of liability litiga-
ory [5], the psychology research conducted by Kenneth and tions to complement risk-based regulations.
Mamie Clark provided justifications. Their famous “doll test” However, given the intricacies of human-AI interactions
demonstrated that “prejudice, discrimination, and segregation” and the multitude of confounding factors at play, the conven-
created a feeling of inferiority among African-American chil- tional fault-based liability system does not work for contempo-
dren and damaged their self-esteem [149]. rary AI-mediated harms. Potential directions include adopting
The California Consumer Privacy Act and the California a strict liability framework that does not require plaintiffs to
Deepfake Law stand as noteworthy examples of legislation de- prove fault, which has been utilized in the EU AI Liability
signed to safeguard human values threatened by algorithmic Directive. Central to this directive is the establishment of a
surveillance and the manipulation of one’s image. These laws rebuttable “presumption of causality.” This provision aims to
draw upon research from diverse disciplines to illuminate the alleviate the burden of proof for victims seeking to establish
concept of privacy harm in the digital era [57, 65, 67, 70, 138]. that the damage was indeed caused by an AI system [1].
For instance, Ryan Calo (2011) delineates two categories of In addition, a “disparate impact” theory developed in rela-
privacy harm: subjective harm, characterized by the percep- tion to the Civil Rights Act of 1964 [19] illustrates possible
tion of unwanted observation, and objective harm, involving direction. This theory means that a seemingly neutral policy
the unanticipated or coerced use of an individual’s informa- or practice could still have a discriminatory effect on a pro-
tion against them [57]. Furthermore, Danielle K. Citron tected group if it leads to significantly different outcomes for
(2019) introduced the notion of “sexual privacy”, which per- different groups [85]. This theory diverges from traditional
tains to the access and dissemination of personal information discrimination laws, which have often focused on intent or

17
explicit discriminatory actions [8]. In particular, the recent (5) Publish summaries of copyrighted data used for training.
settlement between the Department of Justice and Meta [122] Moreover, the current draft includes the creation of the EU
sets a precedent by attributing responsibility to Meta based AI Office to enact and implement universal guidelines over
on acknowledging the disparate impact caused by targeted general- and specific-purpose AI applications.
advertising algorithms [122]. Recognizing the broader impli- In the U.S., federal agencies have developed sector-specific
cations of algorithms in marginalized groups helps address rules for AI use in domains like drug development [42] and po-
the challenges posed by the intricate and unintended effects litical campaigns [2], while overarching initiatives including
of technology on society. the AI Bills of Rights [124] and NIST’s AI Risk Management
Furthermore, courts can utilize affirmative defense sys- Framework [125] aim to provide voluntary guidelines for re-
tems to achieve a balanced approach to liability in AI-related sponsible AI development and deployment. Additionally, an
cases. Affirmative defenses provide AI companies with a agreement between the U.S. government and AI companies
means to demonstrate that, despite unfavorable outcomes, in July 2023 emphasizes safety and security measures in AI
they exercised due diligence, adopted reasonable precautions, development [95]. The U.S. Algorithmic Accountability Act
and followed industry best practices. This approach recog- of 2022 [38] places obligations on companies to conduct im-
nizes the intricate and evolving nature of AI systems while pact assessments, ensure transparency, address performance
upholding corporate responsibility. Consequently, AI compa- disparities, consult stakeholders, report to the FTC, and regu-
nies are encouraged to prioritize the safety of their product larly review and update their Automated Decision Systems to
outputs through strategies like reinforcement learning with mitigate risks and enhance fairness and transparency.
human feedback, red-teaming, and comprehensive evalua- As we have observed in many failed attempts in the field
tion [127, 173]. of online privacy [86], relying solely on the goodwill of cor-
porations is often not sufficient. In the absence of robust legal
and regulatory frameworks, corporate priorities can shift, and
8.3 Comprehensive Safety Regulation market pressures may outweigh commitments to safety and
In addition to traditional legal solutions based on individual security. Although there has been a historical reluctance to
rights and responsibilities, providing step-by-step regulatory enforce stringent tech regulations in the U.S., the intricate and
guidance for those working on AI systems can be a proactive far-reaching consequences of AI-mediated harms suggest a
way to handle potential AI-related problems. By acknowl- potential need for comprehensive legislative measures. The
edging the inherent risks associated with AI technology, this specific design of governance mechanisms can vary, such as
approach facilitates essential measures such as mandatory audited self-regulation, similar to practices in the online ad-
third-party audits of training data, as well as the establish- vertising industry [82], depending on the level of engagement,
ment of industry-wide norms for transparency, fairness, and discretion, and authority of privacy and public entities. These
accountability. This ensures that the industry operates accord- measures must include enforcement mechanisms and provide
ing to recognized guidelines that can help manage risks. This clear guidance and well-defined benchmarks to ensure the
is especially pertinent for Generative AI systems, considering efficacy of the governance.
their potential impact on human values and the swift advances
in aligning AI with these values. 9 Conclusion
The EU AI Act attempts to establish structured safety reg-
ulations for AI across various fields [41]. It initially adopts Generative AI systems present unique and unprecedented
a sectoral approach to define high-risk AI systems that are challenges to human values, including the manipulation of hu-
subject to heightened safety and transparency requirements, man thoughts and the perpetuation of harmful stereotypes. In
based on the types of data and use cases, such as critical in- light of these complexities, traditional approaches within U.S.
frastructure used in healthcare, transportation, energy, and legal systems, whether a gradual case accumulation based
parts of the public sector, and systems that pose risks to indi- on individual rights and responsibilities or domain-specific
viduals’ health and safety. During the legislative process, the regulations, may prove inadequate. The U.S. Constitution
high-profile release of Generative AI systems in 2021 raised and civil rights laws do not address AI-driven biases against
awareness of the capabilities of “foundation models” that op- marginalized groups. Even when AI systems result in tan-
erate without specific intended purposes, and these models gible harms that qualify liability claims, the multitude of
do not fall under the proposed high-risk categories [41]. confounding circumstances affecting final outcomes makes it
Consequently, the EU Parliament introduced a set of re- difficult to pinpoint the most culpable entities. A patchwork
quirements specifically applicable to foundation models as of domain-specific laws and the case-law approach fall short
follows: (1) Comply with design, information, and environ- in establishing comprehensive risk management strategies
mental requirements; (2) Register in the EU database; (3) that extend beyond isolated instances.
Disclose that the content was generated by AI; (4) Design Our analysis supports the need for evolving legal frame-
the model to prevent it from generating illegal content; and works to address the unique and still unforeseen threats posed

18
by Generative AI. This includes developing and enacting laws [11] Reno v. ACLU, 521 U.S. 844, 1997.
that explicitly recognize and protect values and promoting
proactive and transparent industry guidelines to prevent nega- [12] American Manufacturers’ Mutual Insurance Company
tive impacts without placing burdens of proof or causation on v. Sullivan, 526. U.S. 40, 1999.
individuals who are harmed. Achieving ethical and trustwor- [13] Ashcroft v. American Civil Liberties Union, 542 U.S.
thy AI requires a concerted effort to evolve both technology 656, 2004.
and law in tandem. Our goal is to foster an interdisciplinary
dialogue among legal scholars, researchers, and policymak- [14] Anthony v. Yahoo! Inc., 421 F.Supp.2d 1257
ers to develop more effective and inclusive regulations for (N.D.Cal.), 2006.
responsible AI deployment.
[15] Doe v. MySpace, Inc., 528 F.3d 413 (5th Cir. 2008),
cert denied, 129 S. Ct. 600, 2008.
10 Acknowledgements
[16] Parker v. Google, Inc., 422 F. Supp. 2d 492, decision
This work is supported by the U.S. National Institute of Stan- summary aff’d, 242 Fed. Appx. 833 (3d Cir. 2007),
dards and Technology (NIST) Grant 60NANB20D212T and cert denied 522 U.S. 1156, 2008.
the University of Washington Tech Policy Lab, which receives
support from the William and Flora Hewlett Foundation, the [17] Ziencik v. Snap, Inc., Civil Action 21-49 (W.D. Pa.
John D. and Catherine T. MacArthur Foundation, Microsoft, Sep. 8, 2021), September 2011.
and the Pierre and Pamela Omidyar Fund at the Silicon Valley
[18] 18 U.S.C. § 2261A, 2012.
Community Foundation. We thank our colleagues for their
participation as expert panelists: Kaiming Cheng, Miro Enev, [19] 42 u.s.c §§ 2000d - 2000d-7, 2012.
Gregor Haas, Rachel Hong, Liwei Jiang, Rachel McAmis,
Miranda Wei, and Tina Yeung. We thank reviewers of Gen [20] Cullen v. Netflix, Inc. 880 F.Supp.2d 1017 (N.D.Cal.),
Law + AI Workshop at the International Conference of Ma- 2012.
chine Learning 2023 and our colleagues for valuable feedback:
[21] Zhang v. Baidu.Com, Inc., 10 F. Supp. 3d 433
Alan Rozenshtein, Kentrell Owens, Maria P. Angel, Joyce
(S.D.N.Y.), 2014.
Jia, Sikang Song, and our expert panelists. Any opinions,
findings, conclusions, or recommendations expressed in this [22] Constitution of the united states—a history, Nov 2015.
paper are those of the authors and do not reflect those of NIST
or other institutions. [23] O’Kroley v. Fastcase, Inc. 831 F.3d 352 (6th Cir.),
2016.

[24] Regulation (EU) 2016/679 of the European Parliament

References and of the Council of 27 April 2016 on the protec-
tion of natural persons with regard to the processing
[1] Artificial intelligence liability directive. European of personal data and on the free movement of such
Parliament News. data, and repealing Directive 95/46/EC (General Data
[2] FEC approves rulemaking petition, discusses advisory Protection Regulation). https://eur-lex.europa.
opinion. eu/eli/reg/2016/679/oj, 2016.

[3] U.S. Constitution. Amend. XIV., 1868. [25] Packingham v. North Carolina, 137 S. Ct. 1730, 2017.

[4] Neiman-Marcus v. Lait, 13 F.R.D. 311 (S.D.N.Y.), [26] 29 U.S.C. §§ 651-678, 2018.
1952. [27] 42 U.S.C. §§ 7401-7671q, 2018.
[5] Brown v. Board of Education, 347 U.S. 483, 1954.
[28] California Consumer Privacy Act of 2018, Cal. Civ.
[6] Some moral and technical consequences of automation. Code §§ 1798.100 - 1798.199, 2018.
Science, 131:1357, 1960.
[29] 234. Fla. Stat. § 784.048(1)(d), 2019.
[7] 20 U.S.C. § 1681, 1972.
[30] Cal. Civ. Code § 1708.86, 2019.
[8] Washington v. Davis 426 U.S. 229, 1976.
[31] Cal. Penal Code § 528.5(a), 2019.
[9] O’Brien v. Muskin Corp., 94 N.J. 169, 1983.
[32] Facebook to be fined $5bn over cambridge analytica
[10] 47 U.S.C. § 230, 1996. scandal. BBC News, Jul 2019.

19
[33] N.Y. Penal Law § 190.25, 2019. Olah, Danny Hernandez, Dawn Drain, Deep Ganguli,
Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr,
[34] Robles v. Domino’s Pizza LLC, 913 F.3d 898 (9th Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal
Cir.), 2019. Ndousse, Kamile Lukosuite, Liane Lovitt, Michael
[35] Tex. Penal Code Ann. § 42.072, 2019. Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi
Mercado, Nova DasSarma, Robert Lasenby, Robin
[36] Lemmon v. Snap, Inc., 995 F.3d 1085 (9th Cir.), 2021. Larson, Sam Ringer, Scott Johnston, Shauna Kravec,
Sheer El Showk, Stanislav Fort, Tamera Lanham, Tim-
[37] Proposal for a regulation of the european parliament othy Telleen-Lawton, Tom Conerly, Tom Henighan,
and of the council laying down harmonised rules on Tristan Hume, Samuel R. Bowman, Zac Hatfield-
artificial intelligence (artificial intelligence act) and Dodds, Ben Mann, Dario Amodei, Nicholas Joseph,
amending certain union legislative acts, com/2021/206 Sam McCandlish, Tom Brown, and Jared Kaplan. Con-
final, 2021. stitutional ai: Harmlessness from ai feedback, 2022.
[38] Algorithmic Accountability Act of 2022, H.R. 6580, [49] Derek E. Bambauer and Mihai Surdeanu. Authorbots.
117th Cong., 2021–2022. Journal of Free Speech Law, 3, May 2023. Forthcom-
ing.
[39] Digital Services Act: agreement for a transparent and
safe online environment. European Parliament News, [50] Charles R Beitz. Human rights as a common con-
Apr 2022. cern. American Political Science Review, 95(2):269–
[40] Illinois Biometric Information Privacy Act, 740 Ill. 282, 2001.
Comp. Stat. Ann. 14/1 et seq. West, 2022). [51] Emily M. Bender, Timnit Gebru, Angelina McMillan-
[41] AI Act: a step closer to the first rules on Artificial Major, and Shmargaret Shmitchell. On the dangers of
Intelligence. European Parliament News, Nov 2023. stochastic parrots: Can language models be too big? In
Proceedings of the 2021 ACM Conference on Fairness,
[42] Artificial Intelligence and Machine Learn- Accountability, and Transparency, FAccT ’21, page
ing (AI/ML) for Development. https: 610–623, New York, NY, USA, 2021. Association for
//www.fda.gov/science-research/ Computing Machinery.
science-and-research-special-topics/
[52] Aj Blechner. Legal research strategy, December 2022.
artificial-intelligence-and-machine-learning-aiml-drug-development,
May 2023.
[53] Editorial Board. Opinion: Who’s responsible when
[43] Ftc warns about misuses of biometric information and chatgpt goes off the rails? congress should say. The
harm to consumers, May 2023. Washington Post, March 2023.

[44] Gonzalez v. Google LLC. https: [54] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli,
//www.scotusblog.com/case-files/cases/ Russ Altman, Simran Arora, Sydney von Arx,
gonzalez-v-google-llc/, 2023. Michael S. Bernstein, Jeannette Bohg, Antoine Bosse-
lut, Emma Brunskill, Erik Brynjolfsson, Shyamal
[45] Proposal for a directive of the European Parliament and Buch, Dallas Card, Rodrigo Castellon, Niladri Chat-
of the Council on adapting non- contractual civil liabil- terji, Annie Chen, Kathleen Creel, Jared Quincy Davis,
ity rules to artificial intelligence (AI liability directive). Dora Demszky, Chris Donahue, Moussa Doumbouya,
https://www.europarl.europa.eu/thinktank/ Esin Durmus, Stefano Ermon, John Etchemendy,
en/document/EPRS_BRI(2023)739342, 2023. Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor
Gale, Lauren Gillespie, Karan Goel, Noah Goodman,
[46] Todd Rosenbaum Adam B. Korn, Sebastian A. Navarro. Shelby Grossman, Neel Guha, Tatsunori Hashimoto,
An overview of why class action privacy lawsuits may Peter Henderson, John Hewitt, Daniel E. Ho, Jenny
have just gotten bigger – yet again, March 2023. Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil
[47] Sam Altman, Greg Brockman, and Ilya Sutskever. Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth
Governance of superintelligence. OpenAI, May 2023. Karamcheti, Geoff Keeling, Fereshte Khani, Omar
Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna,
[48] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak,
Amanda Askell, Jackson Kernion, Andy Jones, Anna Mina Lee, Tony Lee, Jure Leskovec, Isabelle Lev-
Chen, Anna Goldie, Azalia Mirhoseini, Cameron McK- ent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali
innon, Carol Chen, Catherine Olsson, Christopher Malik, Christopher D. Manning, Suvir Mirchandani,

20
 Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika [65] Danielle Keats Citron and Daniel J Solove. Privacy
Narayan, Deepak Narayanan, Ben Newman, Allen harms. Boston University Law Review, 102:793, 2022.
Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian
Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, [66] Danielle Keats Citron and Benjamin Wittes. The
Joon Sung Park, Chris Piech, Eva Portelance, Christo- internet will not break: Denying bad samaritans sec.
pher Potts, Aditi Raghunathan, Rob Reich, Hongyu 230 immunity. Fordham Law Review, 86:401, 2017.
Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack
[67] Ignacio N Cofone and Adriana Z Robertson. Privacy
Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa,
harms. Hastings Law Journal, 69:1039, 2017.
Keshav Santhanam, Andy Shih, Krishnan Srinivasan,
Alex Tamkin, Rohan Taori, Armin W. Thomas, Flo- [68] U.S. Equal Employment Opportunity Commission.
rian Tramèr, Rose E. Wang, William Wang, Bohan The ada and ai: Applicants and employees, May 2022.
Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michi-
hiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael [69] Ian Cram. The danish cartoons, offensive expres-
Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lu- sion, and democratic legitimacy. Extreme speech and
cia Zheng, Kaitlyn Zhou, and Percy Liang. On the democracy, pages 289–310, 2009.
opportunities and risks of foundation models, 2022.
[70] Kate Crawford and Jason Schultz. Big data and due
[55] Kevin Byron. Creative reflections on brainstorming. process: Toward a framework to redress predictive
London Review of Education, 2012. privacy harms. Boston College Law Review, 55:93,
2014.
[56] Aylin Caliskan, Joanna J Bryson, and Arvind
Narayanan. Semantics derived automatically from [71] Rowan Cruft. Is There a Right to Internet Access?
language corpora contain human-like biases. Science, Oxford University Press, 2022.
356(6334):183–186, 2017.
[72] Anokhy Desai. Us state privacy legislation tracker,
[57] Ryan Calo. The boundaries of privacy harm. Indiana May 2023.
Law Journal, 86:1131, 2011.
[73] Christine M. Drennon. Social relations spatially fixed:
[58] Ryan Calo. Artificial Intelligence Policy: A Primer Construction and maintenance of school districts in san
and Roadmap. University of California, Davis, 51:399– antonio, texas. Geographical Review, 96(4):567–593,
435, 2017. 2006.

[59] Anupam Chander. The racist algorithm?, 115 mich. L. [74] Esin Durmus, Karina Nyugen, Thomas I. Liao,
Rev, 1023:1033, 2017. Nicholas Schiefer, Amanda Askell, Anton Bakhtin,
Carol Chen, Zac Hatfield-Dodds, Danny Hernandez,
[60] Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Nicholas Joseph, Liane Lovitt, Sam McCandlish,
Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared
Xiaolong Chen, Xingmei Wang, et al. When large Kaplan, Jack Clark, and Deep Ganguli. Towards mea-
language models meet personalization: Perspectives suring the representation of subjective global opinions
of challenges and opportunities. arXiv preprint in language models, 2023.
arXiv:2307.16376, 2023.
[75] Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi
[61] Inyoung Cheong. Freedom of algorithmic expression. Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep
University of Cincinnati Law Review, 91:680, 2023. Baines, Onur Celebi, Guillaume Wenzek, Vishrav
Chaudhary, Naman Goyal, Tom Birch, Vitaliy
[62] Danielle Keats Citron. Sexual privacy. The Yale Law Liptchinsky, Sergey Edunov, Edouard Grave, Michael
Journal, pages 1870–1960, 2019. Auli, and Armand Joulin. Beyond english-centric
multilingual machine translation. Journal of Machine
[63] Danielle Keats Citron and Mary Anne Franks. The Learning Research, 22, 2021.
internet as a speech machine and other myths confound-
ing section 230 reform. University of Chicago Legal [76] Nita A Farahany. The battle for your brain: defending
Forum, page 45, 2020. the right to think freely in the age of neurotechnology.
St. Martin’s Press, 2023.
[64] Danielle Keats Citron and Frank Pasquale. The scored
society: Due process for automated predictions. Wash- [77] Noah Feldman. Free speech in europe isn’t what
ington Law Review, 89:1, 2014. americans think. Bloomberg.com, Mar 2017.

21
[78] John Finnis. Natural Law and Natural Rights. Oxford [92] Peter Henderson, Xuechen Li, Dan Jurafsky, Tat-
University Press, 1980. sunori Hashimoto, Mark A Lemley, and Percy Liang.
Foundation models and fair use. arXiv preprint
[79] William W Fisher III. American legal realism. Oxford arXiv:2303.15715, 2023.
University Press, 1993.
[93] Dan Hendrycks, Collin Burns, Steven Basart, Andrew
[80] Giorgio Franceschelli and Mirco Musolesi. Copyright
Critch, Jerry Li, Dawn Song, and Jacob Steinhardt.
in generative deep learning. Data & Policy, 4:e17,
Aligning {ai} with shared human values. In Interna-
2022.
tional Conference on Learning Representations, 2021.
[81] Brian Frederick. AI allows you to talk with virtual ver-
sions of deceased loved ones. Search Engine Journal, [94] Benjamin Herold. The disparities in remote learning
October 2022. under coronavirus (in charts), April 2020.

[82] Jody Freeman. Collaborative governance in the admin- [95] The White House. FACT SHEET: Biden-Harris Ad-
istrative state. UCLA Law Review, 45:1, 1997. ministration Secures Voluntary Commitments from
Leading Artificial Intelligence Companies to Manage
[83] Charles Fried. Privacy: Economics and ethics: A the Risks Posed by AI, Jul 2023.
comment on posner. Georgia Law Review, 12:423,
1977. [96] Leonardo Horn Iwaya, Muhammad Ali Babar, and
Awais Rashid. Privacy engineering in the wild: Un-
[84] Iason Gabriel. Artificial intelligence, values and align-
derstanding the practitioners’ mindset, organisational
ment. Minds and Machines, 30(3):411–437, Sep 2020.
culture, and current practices, 2022.
arXiv:2001.09768 [cs].
[85] David J Garrow. Toward a definitive history of griggs [97] Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior
v. duke power co. Vand. Law Review, 67:197, 2014. Zalmanson, and Mor Naaman. Co-writing with opin-
ionated language models affects users’ views. In Pro-
[86] Robert Gellman and Pam Dixon. Many failures: A ceedings of the 2023 CHI Conference on Human Fac-
brief history of privacy self-regulation in the united tors in Computing Systems, CHI ’23, page 22, New
states. In World Privacy Forum, pages 1–29. World York, NY, USA, 2023. ACM.
Privacy Forum, 2011.
[98] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu,
[87] HH Gerth and C Wright Mills. Politics as a vocation. Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea
From Max Weber: Essays in Sociology, pages 77–128, Madotto, and Pascale Fung. Survey of hallucination in
1946. natural language generation. ACM Computing Surveys,
[88] Sourojit Ghosh and Aylin Caliskan. Chatgpt perpet- 55(12):1–38, mar 2023.
uates gender bias in machine translation and ignores
[99] Hang Jiang, Doug Beeferman, Brandon Roy, and Deb
non-gendered pronouns: Findings across bengali and
Roy. Communitylm: Probing partisan worldviews
five other low-resource languages. Proceedings of
from language models, 2022.
the 2023 AAAI/ACM Conference on AI, Ethics, and
Society (AAAI/ACM AIES), 2023. [100] Liwei Jiang, Jena D Hwang, Chandra Bhagavatula,
[89] Josh A. Goldstein, Girish Sastry, Micah Musser, Re- Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny
nee DiResta, Matthew Gentzel, and Katerina Sedova. Liang, Oren Etzioni, Maarten Sap, and Yejin Choi.
Generative language models and automated influence Delphi: Towards machine ethics and norms. arXiv
operations: Emerging threats and potential mitigations, preprint arXiv:2110.07574, 2021.
2023.
[101] David Jurgens, Eshwar Chandrasekharan, and Libby
[90] Wei Guo and Aylin Caliskan. Detecting emergent in- Hemphill. A just and comprehensive strategy for using
tersectional biases: Contextualized word embeddings nlp to address online abuse. In Annual Meeting of the
contain a distribution of human-like biases. In Pro- Association for Computational Linguistics, 2019.
ceedings of the 2021 AAAI/ACM Conference on AI,
Ethics, and Society, pages 122–133, 2021. [102] Robert A Kagan. Adversarial Legalism: The American
Way of Law. Harvard University Press, 2019.
[91] Claudia E Haupt. Regulating speech online: free
speech values in constitutional frames. Washington [103] Margot E. Kaminski. Regulating the risks of ai.
University Law Review, 99:751, 2021. Boston University Law Review, 103, 2023.

22
[104] Enkelejda Kasneci, Stefan Küchemann Kathrin Sessler, A longitudinal case study on value change in ingrian–
Maria Bannert, Daryna Dementieva, Frank Fischer, finnish migrants. Social Psychological and Personality
Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Science, 2(6):584–591, 2011.
Hüllermeier, Stepha Krusche, Gitta Kutyniok, Tilman
Michaeli, Claudia Nerdel, Jürgen Pfeffer, Oleksandra [116] James Madison. 47. the alleged danger from the
Poquet, Michael Sailer, Albrecht Schmidt, Tina Seidel, powers of the union to the state governments consid-
Matthias Stadler, Jochen Weller, Jochen Kuhn, and ered. In The Federalist Papers, page 209. Open Road
Gjergji Kasneci. Chatgpt for good? on opportunities Integrated Media, Inc., 2022.
and challenges of large language models for education.
Learning and Individual Differences, 103, 2023. [117] Liviu Octavian Mafteiu-Scai. A new approach for
solving equations systems inspired from brainstorming.
[105] Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, International Journal of New Computer Architectures
and Cass R Sunstein. Discrimination in the age of and Their Applications, 5(1):10+, Jan 2015.
algorithms. Journal of Legal Analysis, 10:113–174,
2018. [118] Sandra G Mayson. Bias in, bias out. The Yale Law
Journal, pages 2218–2300, 2019.
[106] Ariel Knafo and Shalom H Schwartz. Value socializa-
tion in families of israeli-born and soviet-born adoles- [119] Elizabeth Mertz, Stewart Macaulay, and Thomas W
cents in israel. Journal of Cross-Cultural Psychology, Mitchell. The New Legal Realism. Cambridge Univer-
32(2):213–228, 2001. sity Press, 2016.

[107] Sarah Knuckey, Joshua D Fisher, Amanda M Klas- [120] Hiroshi Miyashita. A tale of two privacies: enforcing
ing, Tess Russo, and Margaret L Satterthwaite. Ad- privacy with hard power and soft power in japan. En-
vancing socioeconomic rights through interdisciplinary forcing Privacy: Regulatory, Legal and Technological
factfinding: Opportunities and challenges. Annual Re- Approaches, pages 105–122, 2016.
view of Law and Social Science, 17:375–389, 2021.
[121] Ann Nevile et al. Values, rights and concepts of citi-
[108] Noam Kolt. Algorithmic black swans. (4370566), Feb zenship. In Human Rights and Social Policy: A Com-
2023. parative Analysis of Values and Citizenship in OECD
Countries. Edward Elgar Publishing, 2010.
[109] Mason Lawler. State appeals court allows design-
defect claims against snapchat to proceed, January [122] U.S. Department of Justice. United states attorney
2023. resolves groundbreaking suit against meta platforms,
inc., formerly known as facebook, to address discrimi-
[110] Lawrence Lessig. Code version 2.0. Basic Books, natory advertising for housing, June 2022.
2006.
[123] The White House Office of Science and Technology
[111] Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, Policy. Blueprint for an ai bill of rights, 2022.
and Ruslan Salakhutdinov. Towards understanding
and mitigating social biases in language models. In [124] White House Office of Science and Technology Policy.
International Conference on Machine Learning, pages Ai bill of rights, 2022.
6565–6576. PMLR, 2021.
[125] National Institute of Standards and U.S. Department
[112] Cristiano Lima. Ai chatbots won’t enjoy tech’s legal of Commerce Technology. AI Risk Management
shield, section 230 authors say. The Washington Post, Framework: Second Draft, 2022.
March 2023. Analysis by Cristiano Lima with research
by David DiMolfetta. [126] Shiva Omrani Sabbaghi, Robert Wolfe, and Aylin
Caliskan. Evaluating biased attitude associations of
[113] Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Ja- language models in an intersectional context. Proceed-
son Eisner. Decision-oriented dialogue for human-ai ings of the 2023 AAAI/ACM Conference on AI, Ethics,
collaboration, 2023. and Society (AAAI/ACM AIES), 2023.

[114] Natasha Lomas. Who’s liable for ai-generated lies?, [127] OpenAI. Gpt-4 technical report, 2023.
June 2022.
[128] Anne O’Rourke. Caring about virtual perts: an ethical
[115] Jan-Erik Lönnqvist, Inga Jasinskaja-Lahti, and Markku interpretation of tamagotchi. Animal Issues, 2(1),
Verkasalo. Personal values before and after migration: 1998.

23
[129] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- [140] Franklin D. Roosevelt. State of the union message to
roll L. Wainwright, Pamela Mishkin, Chong Zhang, congress, 1944.
Sandhini Agarwal, Katarina Slama, Alex Ray, John
Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, [141] Matthew Sag. Copyright safety for generative ai. Hous-
Maddie Simens, Amanda Askell, Peter Welinder, Paul ton Law Review, 61(2), Aug 2023.
Christiano, Jan Leike, and Ryan Lowe. Training lan- [142] Lilach Sagiv, Sonia Roccas, Jan Cieciuch, and
guage models to follow instructions with human feed- Shalom H Schwartz. Personal values in human life.
back, 2022. Nature human behaviour, 1(9):630–639, 2017.
[130] Kentrell Owens, Johanna Gunawan, David Choffnes, [143] Johnny Saldaña. The Coding Manual for Qualitative
Pardis Emami-Naeini, Tadayoshi Kohno, and Researchers (4th ed.). SAGE Publications, 2021.
Franziska Roesner. Exploring deceptive design
patterns in voice interfaces. In Proceedings of the [144] Shalom H Schwartz. An overview of the schwartz
2022 European Symposium on Usable Security, theory of basic values. Online readings in Psychology
EuroUSEC ’22, page 64–78, New York, NY, USA, and Culture, 2(1):11, 2012.
2022. Association for Computing Machinery. [145] Shalom H Schwartz and Anat Bardi. Value hierar-
chies across cultures: Taking a similarities perspective.
[131] Orlando Patterson. Freedom: Volume I: Freedom In
Journal of cross-cultural Psychology, 32(3):268–290,
The Making Of Western Culture. Basic Books, New
2001.
York, N.Y., revised ed. edition edition, Oct 1992.
[146] Shalom H Schwartz et al. A theory of cultural values
[132] Camille Sojit Pejcha. Tiktok’s “mind-reading” algo- and some implications for work. Applied psychology,
rithm is about to change. Document Journal, Aug 48(1):23–47, 1999.
2023.
[147] Amartya Sen. Elements of a theory of human rights.
[133] Joseph Raz. The Authority of Law. Oxford University In Justice and the capabilities approach. Routledge,
Press, 1979. 2017.

[134] Reuters. Australian mayor prepares world’s first [148] Congressional Research Service. Federal financial
defamation lawsuit over chatgpt content, April 2023. assistance and civil rights requirements. CRS Report,
May 2022.
[135] Neil Richards and Woodrow Hartzog. A duty of loyalty
for privacy law. Washington University Law Review, [149] Richard Severo. Kenneth clark, whofought segrega-
99:961, 2021. tion, dies. The New York Times, May 2005.
[150] Murray Shanahan, Kyle McDonell, and Laria
[136] Kimberly Jenkins Robinson. Designing the legal ar-
Reynolds. Role-play with large language models,
chitecture to protect education as a civil right. Indiana
2023.
Law Journal, 96(1), 2020.
[151] Emily Sheng, Kai-Wei Chang, Premkumar Natarajan,
[137] Kat Roemmich, Florian Schaub, and Nazanin Andal- and Nanyun Peng. The woman worked as a babysitter:
ibi. Emotion ai at work: Implications for workplace On biases in language generation. In Proceedings of
surveillance, emotional labor, and emotional privacy. the 2019 Conference on Empirical Methods in Natural
In Proceedings of the 2023 CHI Conference on Hu- Language Processing and the 9th International Joint
man Factors in Computing Systems, CHI ’23, New Conference on Natural Language Processing (EMNLP-
York, NY, USA, 2023. Association for Computing IJCNLP), page 3407–3412. Association for Computa-
Machinery. tional Linguistics, Nov 2019.
[138] Franziska Roesner, Tadayoshi Kohno, and David [152] Clea Simon. How covid taught america about inequity
Wetherall. Detecting and defending against third-party in education, July 2021.
tracking on the web. In Proceedings of the 9th USENIX
Conference on Networked Systems Design and Imple- [153] Irene Solaiman, Zeerak Talat, William Agnew, Lama
mentation, NSDI’12, page 12, USA, 2012. USENIX Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III
Association. au2, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jer-
nite, Alexandra Sasha Luccioni, Alberto Lusoli, Mar-
[139] Milton Rokeach. The nature of human values. Free garet Mitchell, Jessica Newman, Marie-Therese Png,
press, 1973. Andrew Strait, and Apostol Vassilev. Evaluating the

24
 social impact of generative ai systems in systems and [168] Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin
society, 2023. Caliskan. Contrastive language-vision ai models pre-
trained on web-scraped multimodal data exhibit sexual
[154] Taylor Soper. Microsoft vets lead secretive education objectification bias. ACM Conference on Fairness,
startup using generative ai to help students learn, June Accountability, and Transparency., 2023.
2023.
[169] Chloe Xiang. ’he would still be here’: Man dies
[155] Chase Stokes and Marti Hearst. Why more text is by suicide after talking with ai chatbot, widow says,
(often) better: Themes from reader preferences for March 2023.
integration of charts and text, 2022.
[170] Lixiang Yan, Lele Sha, Linxuan Zhao, Yuheng Li,
[156] Cass R Sunstein. State action is always present. Roberto Martinez-Maldonado, Guanliang Chen, Xinyu
Chicago Journal of Internationall Law, 3:465, 2002. Li, Yueqiao Jin, and Dragan Gašević. Practical and eth-
[157] Cass R. Sunstein. The administrative state, inside out. ical challenges of large language models in education:
(4069458), Mar 2022. A systematic literature review, 2023.

[158] Sonya Thomas. How every student known initiative [171] Cat Zakrzewski. Ftc investigates openai over data leak
will give metro students a victory, March 2021. and chatgpt’s inaccuracy. Washington Post, Jul 2023.

[159] Autumn Toney and Aylin Caliskan. ValNorm quanti- [172] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali
fies semantics to reveal consistent valence biases across Farhadi, and Yejin Choi. HellaSwag: Can a machine
languages and over centuries. In Proceedings of the really finish your sentence? In Proceedings of the
2021 Conference on Empirical Methods in Natural 57th Annual Meeting of the Association for Computa-
Language Processing, pages 7203–7218, Online and tional Linguistics, pages 4791–4800, Florence, Italy,
Punta Cana, Dominican Republic, November 2021. July 2019. Association for Computational Linguistics.
Association for Computational Linguistics. [173] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang,
[160] Kentaro Toyama. Why technology alone won’t fix Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen
schools, June 2015. Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen
Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang,
[161] Eugene Volokh. Academic Legal Writing: Law Review Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu
Articles, Student Notes, Seminar Papers, and Getting Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large
on Law Review. University Casebook Series, 2nd language models, 2023.
edition edition, 2005.
[174] Christopher T. Zirpoli. Generative artificial intelli-
[162] Eugene Volokh. Large libel models? liability for gence and copyright law. Congressional Research
ai output. https://www2.law.ucla.edu/volokh/ Service, LSB10922, February 2023.
ailibel.pdf, 2023.

[163] Jun Wang, Chang Xu, Francisco Guzmán, Ahmed A Threat-envisioning Exercise Material
El-Kishky, Yuqing Tang, Benjamin IP Rubinstein,
and Trevor Cohn. Putting words into the system’s The instruction for the workshop is available at:
mouth: A targeted attack on neural machine transla- https://github.com/inyoungcheong/LLM/blob/main/expert_
tion using monolingual data poisoning. arXiv preprint panel_instruction.pdf.
arXiv:2107.05243, 2021. A detailed overview of the responses obtained is available
at:
[164] James Q Whitman. The two western cultures of pri- https://github.com/inyoungcheong/LLM/blob/main/expert_
vacy: Dignity versus liberty. Yale Law Journal, pages panel_result.pdf.
1151–1221, 2004.

[165] Corey Williams. Appeals court: Detroit students have

a right to literacy, April 2020.

[166] Robin M Williams. American society: A sociological

analysis. New York: AA Knopf, 1970.

[167] Greg Winter. State underfinancing damages city

schools, court rules. The New York Times, June 2003.

25