The Need For Skeptical' Accountants in The Era of Big Data PDF
The Need For Skeptical' Accountants in The Era of Big Data PDF
The Need For Skeptical' Accountants in The Era of Big Data PDF
a r t i c l e i n f o a b s t r a c t
Article history: Big Data is now readily available for analysis, and analysts trained to conduct effective
Received 25 June 2016 analysis in this area are in high demand. However, there is a dearth of discussion in the
Received in revised form 23 December 2016 literature related to identifying the important cognitive skills required for accountants to
Accepted 24 December 2016
conduct effective Big Data analysis. Here we argue that accountants need to approach
Available online 4 January 2017
Big Data analysis as informed skeptics, being ever ready to challenge the analysis by asking
good questions in appropriate topical areas. These areas include understanding the limits
Keywords:
of measurement and representation, the subjectiveness of insight, the challenges of
Big Data
Analysis
statistics and integrating data sets, and the effects of underdetermination and inductive
Informed skepticism reasoning. Accordingly, we develop a framework and an illustrative example to facilitate
Questioning the training of accounting students to become informed skeptics in the era of Big Data
by explaining the conceptual relevance of each of the topical areas to Big Data analysis.
In addition, example questions are identified that accountants conducting Big Data analysis
should be asking regarding each topic. Further, for each topic, references to additional
resources are provided that students can access to learn more about effectively conducting
Big Data analysis.
Ó 2017 Elsevier Ltd. All rights reserved.
1. Introduction
Big Data is characterized by three attributes: velocity, variety and volume (Laney, 2001), and results from two trends:
plummeting cost of storage, and innovative means to analyze and interpret data. These trends have led to exponentially
increasing amounts of data, as every 48 h, as much data is created as was from the dawn of civilization to 2003 (Siegler,
2010). As a result, Big Data is changing business (McAfee, Brynjolfsson, Davenport, Patil, & Barton, 2012) as well as ushering
in a new era for global commerce (Shah, Horne, & Capellá, 2012). That said, the changes needed to accounting education to
better prepare students to effectively utilize Big Data are just beginning to emerge. This paper seeks to help students
understand the need to approach Big Data analyses with the mindset of a ‘skeptical’ accountant.
This paper’s motivation stems from the growing availability of Big Data and how this is shifting professionals’ expecta-
tions of the students they hire. Not only are Big Data skills such as database analyses, visualization creation, and tool selec-
tion required, graduates are increasingly expected to be able to apply higher order thinking skills to related analyses. Shah
et al. (2012) called these Big Data higher order skills Big Judgment. We suggest a key aspect of this Big Judgment is the ability
to adopt an attitude of skepticism by recognizing the need to ask good questions throughout the analyses. To this end, we
identify and classify a preliminary set of seven topical areas, and related questions, into a framework suitable for accounting
⇑ Corresponding author.
E-mail addresses: [email protected] (E. McKinney Jr), [email protected] (C.J. Yoos II), [email protected] (K. Snead).
http://dx.doi.org/10.1016/j.jaccedu.2016.12.007
0748-5751/Ó 2017 Elsevier Ltd. All rights reserved.
64 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
educational purposes. These topics are technology independent and emerge from a deeper awareness of the assumptions
that support Big Data analyses. We expect this list to grow and become more refined in coming years. We incorporated
the topics in our senior level/masters level courses on Big Data—courses outside of the traditional accounting curriculum.
Employers persuaded us to include them as they found many new hires were naïve about important assumptions on which
analysis rests; these new hires, aptly labeled unquestioning empiricists, put too much trust in analysis over judgment (Shah
et al., 2012). As a result, we added these questioning topics to the existing subjects of technology and statistics within our
business intelligence courses.
The paper is organized as follows. We first outline the broad impact of Big Data on accounting and education and then
explain why accounting education needs to produce informed skeptics for this era of Big Data. The learning outcomes
and criteria for our topics are presented next, followed by a presentation of the seven topics. Then, an illustrative example
is presented and referred to throughout the discussion of the seven topics. Conclusions and limitations of the paper follow.
The broad impacts of Big Data on the business environment are technical, managerial and social in nature. They include
how technologies should be selected, how to reasonably limit the scope of an analysis, how to examine the tradeoff of pri-
vacy and security, how to address cultural biases in analyses, and how to address the challenges of privilege and digital
divides (Boyd & Crawford, 2011; Chan & Kogan, 2016). Other issues include ownership of data, and compliance and security
(Gantz & Reinsel, 2012; Kaisler, Mondy, & Coen, 2012). Ritchey (2005) highlights the Big Data management challenge of
structuring socially messy problems that come with incomplete, contradictory and changing requirements. More specific
to accounting, Big Data fundamentally changes how accounting data are understood and reported (Griffin & Wright,
2015). Warren, Moffitt, and Byrnes (2015) claim that Big Data gives firms new opportunities to identify behaviors correlated
with goal outcomes and the ability to formulate, store and analyze new performance measures. They also suggest Big Data
will give firms new opportunities to use external data such as social media sentiment in forecasts. McKinney, Yoos, Green,
and Heppard (submitted for publication) suggest Big Data will create a new consumer driven demand for accounting data
that is atomized, reconfigurable and transparent. A noteworthy concern is the current slow pace of accounting in making
adjustments to meet this new demand (Griffin & Wright, 2015).
These impacts on the business and accounting environment are changing business education (Coyne, Coyne, & Walker,
2016). Most of the suggested changes to educational objectives involve analytical and IT skills, including how to design
queries and data structures, how to prepare data for analysis, how to use statistical and other analytical tools, and how to
create and share visualizations (Chen, Chiang, & Storey, 2012; Kaisler, Armour, Espinosa, & Money, 2013). Further, Lazer,
Kennedy, King, and Vespignani (2014) highlights the need for business students to understand construct validity, scaling
limitations, and reliability and dependencies among data. As noted, Shah et al. (2012) believe business education should pro-
duce students who can also exercise the Big Judgment necessary to appropriately analyze Big Data. In their view, a decision
maker with Big Judgment skills possesses a healthy and informed skepticism that can assess the factors underpinning the
numbers and think critically about the assumptions behind the data. While these informed skeptics have been shown to
improve a wide range of metrics across a variety of business functions, they are relatively few in number; helping more stu-
dents become informed skeptics should not be left to broad educational changes or to technology courses (Shah et al., 2012).
We agree—accounting education also has a role to play in training students to become informed skeptics. While there have
been many calls for modifications in accounting education over the past 30 years in response to changes in the financial envi-
ronment and technology (AAA, 1986; AECC, 1990; Albrecht & Sack, 2000), the presence of Big Data will likely require more
significant changes than ever before, as we can expect an increase in both analytical skills and database skills being added to
accounting curriculum. Finally, not only is Big Data changing how accounting education will be performed, it will also have
an impact on who will analyze the data, given one of the characteristics of Big Data tools is self-service (Gantz & Reinsel,
2012; Lycett, 2013). Krahel and Titera (2015) believe that Big Data will result in more analyses being done by accountants
rather than data analysts; however, Yoon, Hoogduin, and Zhang (2015) suggest that there will not be enough trained
accounting analysts to audit all the available data.
Given the emphasis on data analytics, we suggest accounting education should train students to ask good questions. This
training of higher order thinking skills is consistent with a number of recent directives by accounting education agencies
such as the Accounting Education Task Force, the Pathways Commission, and the Association for the Advancement of
Collegiate Schools of Business (AACSB). Two findings of the Accounting Education Task Force report (Lawson et al., 2015)
are particularly germane to our proposal. First, the task force calls for accounting education to better prepare students for
long-term career demands and reduce emphasis on first job skills. Second, accounting education objectives should reflect
how accountants add organizational value. We believe the ability to ask good questions is a long-term career skill particu-
larly useful to organizational value. Accounting graduates that can ask good questions can improve the quality of their
collaboration and the value of analyses, and hence, long-term value.
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 65
The Pathways Commission calls for graduates to comprehend emerging Information Systems (IS) technologies and to be
able to employ quantitative methods and draw appropriate insights from an analysis. The ability to ask good questions helps
students qualify the insights generated by Big Data analyses. The Pathways Commission also calls for accounting education
to be integrated with other business subjects (Behn et al., 2012). By training students to use questioning skills, accounting
education becomes more integrated with business subjects such as analysis, measurement, representation, IS, and qualita-
tive reasoning. Finally, a recent AACSB accounting accreditation change (Standard A7) calls for more learning experiences
with IS in accounting (AACSB, 2014). By learning to question Big Data analysis, students will be learning to use and evaluate
interpretations from current IS systems.
The value of asking good questions has been demonstrated in a variety of critical thinking studies (Bonk & Smith, 1998);
accounting education manuscripts have called for greater emphasis on training accounting students to think critically for
decades (Baril, Cunningham, Fordham, Gardner, & Wolcott, 1998; Bonk & Smith, 1998; Finley & Waymire, 2013). Critical
thinking has a number of definitions, most of which share common elements such as the ability to work on unstructured
and unfamiliar problems, the acquisition and evaluation of information and knowledge (McPherson, 1996), the ability to
apply rationality and logical thinking to organizational problems, and to examine unstated assumptions (Kimmel, 1995;
Kurfiss, 1988; Scriven & Paul, 1987). While critical thinking in general would improve Big Data analyses, here we focus
on one aspect of critical thinking—asking good questions. While our call for training in the area of questioning skills in
the Big Data arena is, as far as we know, the first in accounting, Boyd and Crawford (2011) in the social science domain also
call for critically interrogating the assumptions and biases behind Big Data. However, most of their insights are directed at
social science issues such as privacy and inequality and do not address aspects of business analyses.
In the absence of an accepted framework or model that would identify key questions for Big Data analysis, we adopted
and modified a model based on semiotics—the study of signs, their significance, and the important role of interpretation
(Peirce, 1907). Semiotics is an appropriate theoretical framework for accounting and analysis as it a general account of all
signs, how signs become linked to objects they represent, and how they are interpreted by particular individuals. On the
semiotic view, accounting produces signs—financial numbers, categories and entries are all signs (Boland, 1993). A sign, also
called a signifier, can be the mark on the page or the act of speech; the object is what is signified, the thing, concept or idea
that the sign refers to. Semiotics has been used elsewhere in accounting literature (Boland & Schultze, 1996; McGoun,
Bettner, & Coyne, 2007) and in our work on information theory (McKinney & Yoos, 2010). In this paper we treat data as signs
and use semiotics to better understand how that data creates unique meaning for each individual. We took the basic outline
of semiotics—creation of a sign, affixing a sign to an object and interpreting that sign—and derived our first three topics that
help identify the assumptions made at each step. We initially used these three in class and subsequently this semiotic
foundation grew to the seven topics presented based on feedback from graduate students, executive training participants,
and conference reviewers and attendees. Throughout this process we maintained the same set of criteria for identifying
useful topics.
4. Learning outcomes
Since a key objective of this study is to improve accounting education and better prepare accounting students, we frame
our questions by a set of learning outcomes that we currently use. For simplicity, they are written here as they would appear
in a syllabus or class exercise.
1. Understand the Difficulties of Effective Analysis. Students often under appreciate the assumptions made in conducting an
analysis.
2. Understand the Limits of Analysis. Students typically overstate the certainty of the results of analysis. For example, com-
mon expressions such as ‘‘the analysis proves” or ‘‘the numbers speak for themselves” indicate far greater confidence in
analysis than is warranted.
3. Flourish in Uncertainty. Most students assume the intellectual posture that problems have a finite or given amount of
information and that analysis leads to right answers. Many academic exercises reinforce this—read a case study and
use the ‘‘information” available to make a decision. In contrast, business decisions involve managing uncertainty and
the overwhelming amount of potential data available, and recognizing all analyses of the data have limitations. Our
educational outcome wants students to appreciate the value of uncertainty in actual business contexts, and the limits
it places on confidence in the results of one’s analysis.
4. Understand the Value of Curiosity. Analytical insight is a creative effort, fueled by curiosity. When students quit being
curious about the data, their ability to conduct informed and skeptical analyses is in jeopardy.
5. Consider Other Data Sources. Students need to be conditioned to ‘‘think outside the box” and not to limit analyses to
traditionally available data, but instead, be willing to become aware of other potential sources of data that may have
relevance.
6. Consider Other Approaches to the Analysis. Students need to be made aware of heuristic-based decision making biases (e.g.,
anchoring, status quo, sunk cost, confirming evidence) presented by Hammond, Keeney, and Raiffa (1998) when conduct-
ing analyses. Using alternate approaches to the analyses with other data sources noted above, would assist in mitigating
the effect of these biases.
66 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
7. Evaluate the Costs/Benefits of the Analysis. Given the magnitude of Big Data available, students must be trained to be ever
aware of the cost/benefit tradeoff of revising, expanding, stopping, and perhaps even abandoning the analysis.
Given the plethora of topics related to Big Data that would potentially generate good questions, we imposed criteria our
selected topics must meet in order to be included in the framework. These criteria reduced the number of topics discussed to
seven.
First, each topic should be applicable to most Big Data applications in any business context. Second, each topic is tech-
nology agnostic, which permits a focus only on higher order questioning skills that can be applied to any technology. Third,
each topic addresses the higher level educational objectives of evaluate and create (Anderson & Krathwohl, 2001; Bloom,
1956). Fourth, no topic requires advanced knowledge or practice. We eliminated other topics that require a technical back-
ground or more training in communication or visualization techniques.
6. Topics
The seven topics are summarized in Table 1. We provide an illustrative example, describe each topic in turn and include
within each topic a reference to the illustrative example, where to read more about the topic, and examples of questions.
These seven topics are placed within three phases that follow the analytical flow of a general information process model,
from measurements that constitute data to interpretation conceived from those data to combining and analyzing the data
to generating results and explaining them.
The example in Tables 2 and 3 present a situation common to managers—assessing the economic viability of various pro-
duct lines. The focus of this example is intentionally on managerial accounting, as much of the recent literature discussing
the implications of Big Data for accounting have emphasized impacts on financial accounting and auditing (see Griffin &
Wright, 2015 for a good overview). That said, references will also be made to the financial accounting and auditing implica-
tions in the ensuing discussion of the seven topics, as well as in the classroom Implementation Guidance section.
The Scio Division of Georgetown, Inc. manufactures and sells four related product lines. Each product is produced at one
or more of the three manufacturing plants of the division. Following is a product-line profitability statement for the recent
year ended December 31 that shows a loss for the Baseball Equipment line (BBE). A similar loss for this line is projected for
next year.
Baseball equipment is manufactured in the Evanston Plant. Some football equipment and all miscellaneous sports items
also are processed through this plant. A few of the miscellaneous items are manufactured, and the remainder are purchased
for resale. The items purchased for resale are recorded as materials in the records. A separate production line is used to pro-
duce the products of each product line. The following schedule presents the costs incurred at the Evanston Plant this past
year. Inventories at the end of the year were substantially identical to those at the beginning of the year.
The management of Georgetown, Inc. has requested a profitability study of the Baseball Equipment (BBE) line to deter-
mine if the line should be discontinued. [note: ‘‘unquestioning empiricists” would suggest the above is sufficient to conclude the
line should be dropped].
In addition to the financial data, Georgetown has purchased data from two Big Data sources. The marketing department
has generated a sentiment analysis score from social media of 0.7, a new low. BBE has also purchased data about sourcing
suppliers that is unstructured text—including published financial analyst recommendations, litigation or various forms of
Table 1
Seven questioning topics for big data in accounting education.
Table 2
Georgetown Inc product line profitability YTD December (000’s omitted).
Football equipment Baseball equipment Hockey equipment Misc. sports items Total
Sales $2200 $1000 $1500 $500 $5200
Cost of goods sold
Material $400 $175 $300 $90 $965
Labor and variable overhead $800 $400 $600 $60 $1860
Fixed overhead $350 $275 $100 $50 $775
Total cost of goods sold $1550 $850 $1000 $200 $3600
Gross profit $650 $150 $500 $300 $1600
Selling expense
Variable $440 $200 $300 $100 $1040
Fixed $100 $50 $100 $50 $300
Corporate administration expenses $48 $24 $36 $12 $120
Total selling and admin. expenses $588 $274 $436 $162 $1460
Profit contribution $62 $ (124) $64 $138 $140
Table 3
Evanston plant costs-past year (000’s omitted).
insolvency protection and media comments—in order to generate advanced warning of suppliers’ financial distress. This
measure is called the supplier risk analysis score. A similar risk analysis score was obtained for customers.
In the presentation of the seven topics that follow, the constructs embodied by the topic will be explained and then
related to the BBE example. Big Data implications for the topic will be discussed followed by a listing of questions that
informed skeptics should consider asking. Each topic discussion concludes with a listing of additional resources students
can access.
6.2. Evaluate and enumerate the assumptions which were made in selecting and interpreting measurements. Assess their reliability
and overall value, seek better measures if feasible
Big Data analysis starts with measurement and the assumptions underlying all measurements are typically underappre-
ciated (Stonebraker & Hong, 2012). Measurements travel well—they are easy to create, store and transmit and they enjoy
wide acceptance. The importance of measurement is evident in the adage ‘‘You can’t manage what you can’t measure”
(McAfee et al., 2012). However, every measurement requires assumptions and these assumptions are often overlooked.
Measurement is the process of assigning numbers or other symbols to things and typically assigns numbers to represent
magnitudes of pre-existing quantities. While text, dimensions, and other non-quantitative measures can be used in Big Data
analysis, much of the data are quantitative measures.
Several distinctions about measurement are important to understand since each requires different assumptions. Not all of
these distinctions can be presented in a limited article; here we will discuss three to raise general awareness. First, measures
are either direct (also called fundamental) or indirect (or derived). This direct/indirect distinction should not be confused
with direct and indirect accounting methods. A direct measurement means an attribute of an object can be inspected imme-
diately, for example you can actually pick it up and measure the sides, such as a box, or count items, such as an inventory. An
indirect measure is something like the volume of a rock, where you need a procedure or technique to measure it, like drop-
68 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
ping a rock in a container with water, to determine how much the water rises. Surveys, test results, assessments of invoices
and subjective ratings are examples of indirect procedures in business. The procedure for creating an indirect measure intro-
duces assumptions, often overlooked, including how a survey was constructed, how ratings were combined, how non-
responses were measured, and why a particular procedure for inputting data on an invoice was selected.
A second distinction classifies measures as either real or abstract. A real measure represents a physical attribute of an
object such as size or weight. An abstract measure is not related to a physical property, but is an abstraction such as depre-
ciation, correlation, or rank. As with indirect measures, abstract measures introduce assumptions that are often taken for
granted, such as why a method to calculate depreciation was chosen, how correlation was tested, or why a particular
response scale was chosen.
A third distinction arises from the classic taxonomy of measurement levels: nominal, ordinal, interval and ratio (for addi-
tional background see: Frankfort-Nachmias & Nachmias, 2007; Stevens, 2012; Velleman & Wilkinson, 1993). Each level
encompasses all prior levels; i.e. ordinal is also nominal, interval is also ordinal and nominal, and ratio is also interval, ordinal
and nominal. Nominal measures express ‘‘names” or ‘‘labels” only—these may be expressed in numerals as well as other
symbols, but only ‘‘counts/frequencies” and modes may be reported. Ordinal measures provide further information regard-
ing the order of preferences which may be numerically ranked—but only modes and medians can be reported. Interval mea-
sures assume the ability to quantify distances between points on a scale, permitting their addition, subtraction, and
descriptive measures for central tendency and variation to be reported. Given the popularity of the use of interval measures
for surveys via Likert scales, the assumption that true interval measurement has been achieved is often glossed over. For
example, Likert responses of ‘‘strongly-agree,” ‘‘agree,” and ‘‘neutral” may be assigned scores of 2, 1, and 0 respectively,
but is it valid to assume a sample average of 1.5 is half-way between agree and strongly agree? Finally, ratio measures
are considered the ultimate measure since they include a ‘‘true” zero point, permitting further mathematical computations
of multiplication and division. This notion of a true zero point is illustrated by the Kelvin temperature scale, which includes a
true zero point representing the absence of molecular motion.
Most accounting tasks require the application of measures for decision making. For example, a capital investment deci-
sion may require indirect and abstract measures of sales, operating costs, required inventory levels, and the tax implications
of depreciation. To conduct the analysis, the accountant must be familiar with the assumptions underlying the procedures to
create each of these measures and why those measures were chosen.
1. What indirect and direct measures are available, and what is explicitly known about the measurement (who did it, how
was it done, what instrument/technology was used, how was it manipulated)? What assumptions were made, and what
unknowns can be investigated for better understanding? What about the measurement may never be known, and what
level of uncertainty or loss of value results from this unknowability and uncertainty?
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 69
2. Identify and classify all measures. Which are direct measures? Was the most direct, explicit, reliable measure used (if not,
what would it have been)? Which are abstract measures? What type of abstraction was used and what loss of resolution
and reliability results (i.e. what type of error or uncertainty was, or might have been, introduced)? For each measure:
What is the value and reliability of this measure?
3. Based on a priori knowledge of the goal for this data analysis, list alternative measures which could add value (select these
based on the known goals, and decisions to be made).
4. Measurements of individuals (surveys, analysis of feedback, comments, etc.) are always indirect and often abstract. Eval-
uate the procedure used to generate the measure, why that procedure was used, and how these choices impact the reli-
ability and usefulness of the measure. Further, determine if the individuals measured were aware of how they were being
assessed and evaluate how this Hawthorne effect might impact the usefulness of the measurement.
5. Measures are not the phenomenon or object being measured; they reflect, objectify or interpret some aspect of the object.
Is the measurement of the attribute being mistaken for the inherent object itself?
6. Explain which assumptions can be tested or validated. Once validated, can they still be useful and reliable – to what
degree was the needed, desired or intended measurement level satisfied? What can be done to improve the validity of
measurements?
1. For a good introduction to the topic of measurement: Allen & Yen (2001). Introduction to measurement theory. Waveland
Press.
2. A classic, dated, but available treatment for beginners: https://suppes-corpus.stanford.edu/techreports/IMSSS_45.pdf.
3. A recent academic article that questions the objectivity of accounting measures: http://www.cluteinstitute.com/ojs/in-
dex.php/IBER/article/viewFile/1799/1779.
6.3. Understand and enumerate the assumptions which were made by individuals when they affix a sign (or measure) to an object
to create an interpretation. Evaluate the suitability of the sign for the object, determine if other sign-object combinations are better
In topic 1 we identified questions about the measure, the sign. Here we take the next step and seek to uncover assump-
tions about how those signs are affixed to objects. For a sign (a measure) to mean something to someone, a sign must be
affixed to an object. In semiotic theory a representation is a sign-object combination for an observer (Floridi, 2005;
Zoglauer, 1996). A sign represents, or signifies, something else (Beynon-Davies, 2009; Stamper, 1985; Vigo, 2011). For exam-
ple, dark clouds (sign) portend a storm (object) to a hiker (observer). Similarly, when an end user (observer) examines an
icon (sign) of a dollar bill (object); the icon is representation about the dollar. Common accounting signs that are affixed
to objects include transactions, financial statements, and interest rates. All accounting measures are signs that signify real
objects or events to an observer.
However, in use, a sign-object combination regularly becomes a less specific, less rigorous idea. While the theory requires that
a sign, object, and observer be specified; often only a sign is specified; the object and observer are assumed. The combination
becomes the ring of the doorbell, the dark clouds, the interest rate, or the count of inventory. When individuals do this, they fre-
quently assume the object that they believe the sign represents is known by all. However, in many situations, individuals affix the
sign to different objects which leads to different interpretations (Boland, 1987; Checkland & Holwell, 1998; Dretske, 1981;
MacKay, 1969). The sign never comes with the object, it must be affixed to the object by an individual, but the individual often
takes this step for granted. The sign cannot tell an analyst the right object to affix it to—that requires judgment.
Data are signs and by themselves data do nothing. Only when individuals affix the data to an object is an interpretation
created, minds are persuaded and decisions are made. Analysts should be capable of examining how and why signs are
affixed to objects, and ask good questions about this necessary, but often overlooked choice. Signs and objects are the foun-
dation for interpretation in accounting (Bisman, 2010; Boland, 1987; O’Leary, Wilson, & Metiu, 2014; Previts & Merino, 1979;
Ritson, 2001). For a managerial accounting example, consider the common problem among students when challenged to
identify the root cause of a problem. A teacher might call these different objects symptoms, and direct the students to affix
the sign to a different object, what the teacher feels is the root cause. Without a teacher’s ‘‘right answer,” analysts often affix
signs of data to different objects without realizing that other analysts make other sign-object choices, and that their com-
bination, like the student who affixed the sign to a symptom rather than the root cause, could be improved.
each of these sign-object combinations. Further, it should be asked if there are other data and procedures that make one of
these three sign-object combinations, or others, more useful and reliable. The data, the signs, signify different insights; the
skeptical accountant realizes each accounting number can signify different things when it is attached to different objects.
1. Why do we think this sign is appropriate to affix to this given object? What are the disadvantages in using this specific
sign for the given object, and what are the advantages? What are the cost/benefit tradeoffs for this sign-object
combination?
2. How strongly do other individuals support using this sign for that object? What other objects are other analysts affixing
this sign to? Why has someone chosen to address this object with this sign—is it convenience and availability of the sign,
is it convention, or is it how other companies do it, or some other explanation?
3. In what ways do we attribute this sign to this object out of tradition? In what ways do we attribute this sign to this object
by common cultural bias? For what other reasons did we pick this sign?
4. What sign could be created from the data that would be better for that object? What can we do to acquire and use those
alternative signs? Are there other strategies we could employ to uncover how different individuals are assigning signs to
objects– what are those?
5. Based on the goal or objective of the analysis, are we, or I, or any decision maker focusing on the right objects? What
objects should be addressed? Compare the available signs for these objects, which are most useful? What other objects
might be better, or as good, to focus on?
6.4. Be able to evaluate the extent to which a given representation is unique to each individual. Be able to identify where these
intersubjective differences are significant or unnoticed and be able to use these differences in an informative way
In the previous topic, if individuals affix different signs to different objects different interpretations (or meanings) are cre-
ated. Even if two individuals use the same sign-object combination, the interpretation they create may still be different
(Chandler, 2007; Hong, Morris, Chiu, & Benet-Martinez, 2000; Shore, 1996). For an interpretation to be the same, and intersub-
jective agreement to occur, not only must the sign and object be the same, the interpretation of the sign and object must be very
similar for each individual. Differences in interpretation can be minor and intersubjective agreement is reasonably assumed. At
other times, these differences can be significant, surprising, useful, and informative; they are also normal and frequent. Most of
the trouble caused by these differences arise when they go unnoticed. The impact of unnoticed interpretations has been the
focus of behavioral decision theory and cognitive psychology (Kahneman & Tversky, 1979; Schwenk & Valacich, 1994;
Tversky & Kahneman, 1974), with these studies consistently showing that cultural biases and individual interpretation are
common.
Accounting tasks can be affected by significant or unnoticed differences. For one accountant, a particular transaction
(sign) is an exception (object), but needs no further examination. For another accountant, the same transaction is also an
exception, but further investigation is warranted. In a second example, the two accountants could also derive different inter-
pretations about the risk of a project even though they use the same numbers (signs) for the relevant factors (objects).
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 71
1. How unique is my interpretation of the sign-object? What interpretations are others creating from this sign-object? What
are the advantages and disadvantages, and the costs and benefits of the various interpretations?
2. Assess the extent to which experience, tradition, or culture is leading different individuals to unique interpretations. Eval-
uate the extent to which interpretations are flexible and can become more uniform with communication or they are more
fixed.
3. To what extent do other individuals recognize their own unique interpretation?
4. How can I create the sign-object interpretation that others use? By making this interpretation, what other interpretations
become clear?
5. On which interpretations is a speaker unaware of a lack of intersubjective agreement, how can this lack of agreement be
best addressed?
6. To what extent, and on which sign-object combinations does the speaker or decision maker treat their own interpreta-
tions as fact, as objective? How can you move this individual away from this objective viewpoint?
1. An explanation of fact and opinion and the distinctions between them: http://www.philosophersmag.com/index.php/
tpm-mag-articles/11-essays/26-the-fact-opinion-distinction.
2. An academic article on the assumptions of intersubjectivity in accounting: http://www.sciencedirect.com/science/article/
pii/036136829190025A.
3. A good introduction on representations in semiotics: http://visual-memory.co.uk/daniel/Documents/S4B/sem02a.html.
72 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
6.5. Be able to identify potential differences in databases that have been combined in this analysis, the types of transformations used
to integrate them, and the assumptions and possible errors that result
One of the fundamental characteristics of Big Data is that the data often comes from a variety of sources. These data often
vary on a number of attributes such as completeness, currency, credibility, precision, auditability, objectivity, reliability, and
trustworthiness (Šķiltere & Jesiļevska, 2014; Merino, Caballero, Rivas, Serrano, & Piattini, 2015). While these attributes are
important to consider when analyzing a single database, they become even more important to recognize when databases
are combined. Databases vary on these attributes and in order to combine them, corrections and transformations of the data
are necessary. A common example is combining data from different political polls—how to combine polls of likely voters and
all voters, or polls that allow undecided with those that don’t.
In addition to variation in data quality attributes, other differences in databases may create errors. The data may have
been obtained at different times, for different objectives, or with different metadata. Further, changes in the technology used
to gather and store the data, in data types, in security used, and the data architecture can also lead to incompatibility issues.
Finally, the taxonomies or classifications used to generate the data in different databases may have different definitions.
Accountants often encounter the issue of data variation, or data incompatibility when asked to roll up cost measurements
that were generated using different rules. They also make profitability assessments by allocating costs across different prod-
ucts or divisions even though what should count as costs differ in each product or division. Further, the EY Reporting: Issue 9
(2015) describes these and other barriers to data integration as it pertains to the audit profession.
1. In the databases that were combined for this analysis, compare and contrast the data quality issues of completeness, cur-
rency, credibility, precision, auditability, objectivity, reliability, and trustworthiness. How do the various data sources
vary on these dimensions, and to what extent do these differences limit the analysis?
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 73
2. Where transformations of the data were necessary? Identify the rules used to transform the data and assess the costs and
benefits of that transformation. Evaluate how the transformations impacted subsequent analysis.
3. Evaluate whether subsets of the data were obtained at different times, for different objectives, with different meta data,
with different technology, or using different classifications. Enumerate the ways in which these differences impact sub-
sequent analysis.
4. For each set of data used in the analysis, determine whether data was eliminated during the combination and assess the
impact of the elimination of this data on the results?
5. If any of the above questions are answered in the affirmative, who decided to alter or transform the data, what other
transformations were appropriate but ruled out? Attempt to determine why that data was altered.
6. Why was each of the databases collected and to what purpose is it typically used? Determine if that purpose or goal
impacts the quality of the data, or its usefulness when combined with other data.
6.6. Be aware of common statistical errors, be able to critique the misapplication of statistics to Big Data databases, and be able to
describe limitations of statistical analysis
The misuse of statistics has been well identified in the business environment by books such as ‘‘How to Lie with Statistics”
(Huff, 1993), and by commonly used expressions such as ‘‘Lies, Damn Lies, and Statistics” (Twain, 1906). Common examples
of statistical errors include correlation not causation, non-representative sampling, scaling distortions, elimination of unfa-
vorable data, mistaking correlation for explanation, confusion between samples and populations, and others.
construction errors, communication breakdowns, etc. Accordingly, analysts need to understand and appreciate statistical
errors and misuse and how they limit the validity of Big Data analysis.
1. Evaluate if correlation is being treated like causation. What other factors may be affecting both factors in the correlation?
Do other individuals interpreting this data assume causation?
2. If sampling was conducted, enumerate the ways that the sample is not representative of the total population. Which of
these ways is significant and what impact does that have on the results?
3. Identify each choice that was made in the scaling of the results’ presentation—were these choices made to make the
results from the analysis stronger than warranted? Specify other choices that could have been made, for example in
the scaling of the data, and evaluate if those presentations would have been more appropriate? What is the data being
compared to, what could it have been compared to and why was the selection made?
4. Evaluate the rules used to generate the sample and identify where choices were made. Create examples of data that these
rules would have eliminated and assess if this type of data should have been excluded.
5. Assess the validity of the rules used to categorize discrete data (if dates were used why these dates, if four categories of a
variable, what impact would there be with five or six categories, etc.?)
6. If an entire population was used, rather than a sample, evaluate how exceptions to the population were treated—are some
treated as noise and dismissed that should be retained and evaluated? In population analyses are the limitations of mea-
surements and variation in collection techniques adequately considered?
1. 1. The updated classic volume on how statistics can be misused: Huff, D. (2010). How to lie with statistics. WW Norton &
Company.
2. A good overview of the challenges of Big Data on several key statistical issues: Bollier, D., & Firestone, C. M. (2010). The
promise and peril of big data (p. 1). Washington, DC: Aspen Institute, Communications and Society Program.
3. A well written account of the perils of letting the data speak without an a priori theory: Harford, T. (2014). Big data: A big
mistake? Significance, 11(5), 14–19.
6.7. Be able to apply the lessons of underdeterminism and its implications for analytics
The final two topics shift the focus from data to results; more specifically, they apply to the explanation of results. For the
purposes of our discussion, an analysis is the examination of data using a procedure or statistical technique that generates
one or more results. The explanation then occurs via interpretation of results, which are primarily quantitative in nature.
Relevant to this process of results explanation is the area of underdetermination of scientific theory, also called underdeter-
minism. This is a concept in philosophy of science that states the evidence available at a given time may be insufficient to
determine what beliefs should be held in response to it (Babich, 1993). In other words, for any set of data, a theory that
explains patterns in the data underdetermines the data, there will always be other, at least equally plausible, explanations
for the observed patterns. Underdetermination was first articulated by John Stuart Mill, ‘‘an hypothesis is not to be received
as probably true because it accounts for all the known phenomena, since this is a condition sometimes fulfilled tolerably well
by two conflicting hypotheses. . .while there are probably a thousand more which are equally possible, but which our minds
are unfitted to conceive” (Ducheyne, 2008; Mill, 1867). Quine applied this insight to all knowledge claims (Quine, 1990).
This epistemological challenge in science has profound implications for confidence that any theory will provide the truth
about a set of data. But as Quine pointed out (1990), this challenge is not limited to scientific theories, but also extends to any
results about any set of data. One way epistemologists explain this is with the metaphor of map making. Many ‘‘true” maps
can be created for a territory because many display distinct or different attributes about the territory such as resources, roads
or populations. Disconcertingly, underdeterminism goes one step further—the maps can contradict each other and still be
‘‘true.”
Underdetermination occurs in accounting, for example, audits apply rule sets to a set of transaction data to spot anoma-
lous patterns. Different rule sets applied to the same data will uncover different anomalies. Underdetermination tells us that
there will always be other audit rule sets that can be applied to the same set of transactions that will also produce at least as
many possible anomalies.
from the Big Data analysis needs to be carefully reconsidered in light of potential competing relationships suggested by the
other data analysis findings. For example, a deeper analysis of the ‘‘low” customer sentiment score might reveal a pattern of
customer complaints regarding the quality of some of the leather used in our baseball gloves, which is the dominant product
in this line. This in turn could explain the lowering of prices in response to the reduced demand as a result of this quality
problem. Or, an analysis may indicate a pattern of new competition using cheaper offshore labor which is taking some of
our market share, prompting price reductions. Accountants should resist the temptation to over-ascribe confidence in any
single explanation emerging from a Big Data analysis, as there are likely limitless other explanations. A cost/benefit analysis
must be evoked to assist in determining when to stop searching for alternative explanations to the one settled upon from the
Big Data analysis. Information economics is at play here, as both the marginal cost, and marginal benefit, of obtaining further
information regarding the sign-object-interpretation relationship must be considered.
1. After generating a result from analysis of the data, enumerate other results for other analysis of the same data that are as
plausible as the one presented. Further, generate contradictory results that may also be true.
2. Evaluate the advantages and disadvantages of continuing to look for other results. Determine an appropriate stopping
rule for discontinuing this effort to find other results from other analysis.
3. For explanations of results given to me by others, evaluate the strength and conviction they have in the validity of their
results. If warranted, determine other explanations of the data. Evaluate if others are overconfident about the current
results and point out other results.
4. In what ways has our overconfidence in our results truncated our discussion of the limitations of this analysis; determine
the factors that have led to the premature acceptance of the current results.
5. Identify other missing data that if available might show the current results to be less reasonable. Identify other factors, or
other data that if you had collected it, might also account for the evidence. Specify unquantifiable factors that were
ignored that would change the results.
6. Craft an appropriate result statements while avoiding stating these results too conclusively. Create a balanced statement
of the results that appropriately qualifies the result without gutting it.
6.8. Be able to explain induction and the limits it creates for analytics
With induction, or inductive reasoning, premises lead to conclusions, and conclusions are never certain, only probable,
even if all the premises are true. Induction reasons from the particular to the general. In contrast, deductive reasoning, is
the process of reasoning from many statements to reach a definitive conclusion. With deductive reasoning, if the statements
are true, the conclusion must also be true. It argues from general to particular. An inductive example is the statement that all
transactions we have audited are valid, therefore all the firm’s transactions are valid. A deductive example is all transactions
during this hour were recorded on device x, and device x is certified; therefore, all transactions during this hour are certified.
All accounting documents and data are premises which lead to inductive explanations. No number of fraudulent trans-
actions proves fraud; no history of reasonable journal entries verifies truth and removes all doubt. All conclusions, verifica-
tions and attestation are tenuous. One implication for accounting is that because explanations cannot be true or proven, but
only reasonable, and as a result, determining how much data or evidence is required to establish reasonableness becomes
the fundamental judgment issue. Accounting students need to understand how these judgments are made, how they can
be improved, and how to collect the best evidence for this standard. By deduction we know that if a given Big Data analysis
is inductive, and all inductive explanations (or conclusions) can be wrong, the explanations of the Big Data analysis and
76 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
results can be wrong. This logic should lead accounting students to conclude that every inductive explanation can always be
wrong.
1. Identify and enumerate where the analyst is qualifying the strength of the explanations appropriately. Specify where the
careful identification of the limits of the analysis and the limits of the explanations can be found.
2. Specify examples of data that would disprove or contradict the explanation, at least five. Analyze the likelihood for each of
these five data points occurring.
3. Articulate the rule being used to stop collecting data, and ask, why was this rule used? Evaluate the advantages and dis-
advantages of obtaining more data.
4. Given that an explanation cannot be proven, how much data will be needed to achieve a reasonable amount of confidence
in an explanation?
5. An inductive argument often generalizes from the particular to the general; a pattern found in domain x applies to the
larger domain y. Specify the differences between the particular domain x where the data was generated and the larger
domain y to which the explanations are applied. Evaluate and identify which of these differences is significant.
6. Given the limitations of induction, what language should be used to best express the limitations of the explanations?
7. Implementation guidance
We have taught these topics to undergraduate and graduate business analytics and accounting students over the past four
years. We suggest these exercises be used throughout the second half of the course rather than early in the semester. Then,
these lessons can be applied to the analysis and projects the students completed in the first half of the course. The overall
goal of helping students build an awareness of the many assumptions that underlie Big Data analysis is achieved via repe-
tition—each week a different topic of assumptions is discussed. We have also found that these topics work well when they
are assigned to teams of students. The students research a topic, find appropriate examples from earlier class exercises or
from analysis outside of class, and present their work to their classmates. We asked the teams to generate a short list of
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 77
two or three questioning principles for each topic that then were evaluated on the final exam. On the final exam, we provided
an example analysis and set of visualizations and asked the students to apply the seven topics to identify the most significant
limitations to the analysis.
The BBE example can be used to accomplish many of the previously specified learning outcomes. The suggested use of
this example involves the following sequencing of steps: (1) Provide students only the information originally presented
in Table 2 (this excludes Table 3, the discussion of acquiring Big Data, and the additional information in Appendix A). Then,
facilitate a discussion regarding the reported loss of $124 and the assumptions required to conclude this entire loss could be
avoided if the BBE line was eliminated. Of course, the assumptions would be that all BBE costs would be avoided if elimi-
nated, no impact on revenues/costs of other lines, no economic opportunities of alternative uses of freed resources, etc.;
(2) Discuss what other data sources would be useful in assessing the viability of these assumptions, which leads to the infor-
mation provided in Table 3, and the additional information provided in Appendix A, being presented to students; (3) Require
students to modify the initial ‘‘sign” of the $124 loss to a more accurate assessment of the annual profitability impact of elim-
inating the BBE line via Table 4.
Steps (1)–(3) above will simply ground students with a familiarity of the ‘‘traditional” approach to the analysis conducted
in this decision context. The following steps will expand the discussion by highlighting the potential use of incorporating
‘‘non-traditional” Big Data into the analysis. (4) Ask students if the modified analysis in step (3) is sufficient, or are other data
available to further refine the analysis. To facilitate the discussion, students can be made aware that internal and external
benchmarking suggests that sales are underperforming and costs are out of line. (5) Ask students if they should question
the benchmarks used—are they realistic, how were they developed, what assumptions were used, etc.? (6) Also, inform stu-
dents of the following Big Data obtained: marketing operations has a sentiment analysis score from social media of 0.7, a
new low; data about key material suppliers that is unstructured text—including published financial analyst recommenda-
tions, litigation or various forms of insolvency protection and media comments—in order to develop a supplier risk analysis
score; similar data for customers is also obtained in order to develop a customer risk analysis score. (7) Ask students if they
feel this additional data is germane, and if so, what does it tell us and how can it be used? For example, what data is available
to further explain the low sentiment analysis score—poor quality, delayed shipping, poor customer service, stale product
designs, etc.? What specific problems generate the supplier and customer risk scores if high risk is indicated—operational
problems, cash flow concerns, lawsuits, product quality, etc. (8) Further, ask students how the Big Data should be incorpo-
rated into the analysis regarding the BBE line. Should it impact assumptions for future operating and financial budgets, indi-
cate more research and development is needed for product design, indicate changes are needed to the supply chain structure
for materials, invest more in selling and administrative functions, etc.? (9) Should other Big Data sources be considered, such
as regional macroeconomic, demographic, weather pattern, rates of youth participation in school or community baseball
activities, etc.? (10) And finally, how do we determine at which step above to stop and make a decision given needed con-
siderations for the costs/benefits of obtaining more data?
A nice way to wrap up this classroom discussion is to move into the financial accounting and audit implications of this
example. Augmenting the traditional data with Big Data could assist in the valuation of allowance for doubtful receivables,
inventory, and reserves for customer returns which would obviously impact the income statement and balance sheet. Fur-
ther, auditors may augment or extend traditional audit evidence with Big Data to assess the reasonableness of the estimates
used for valuing the assets and reserves mentioned above, as well as to assist with assessing client risk. Once these issues
have been discussed about BBE, instructors can apply the seven topics to this BBE case and ask students to generate a list
of their own questions within each topic that would apply to any Big Data case with this context.
8. Limitations
While our goal is to help students understand assumptions underlying Big Data analysis, we are not questioning the need
to conduct effective analysis. We simply are echoing calls from the workplace that our students not only be able to conduct
analysis, but act as informed skeptics and understand the limits of the analysis. And as the era of Big Data matures, more
topics may be identified and examined and added to our structured list. As experience with Big Data grows, other key ques-
tioning skills are sure to emerge. They may involve privacy, how to manage iterative analysis, how to state a clear problem or
question and gain buy-in from stakeholders, how to articulate or explain the assumptions or uncertainty of the data to the
data consumers. Finally, we recognize one key limitation to our topics: analysts often do not have complete access to how
measurements were collected for the data they are analyzing.
While Big Data analysis may appear to remove ambiguity, we believe it exacerbates it. Some questions are answered but
just as many more are created. Understanding the ambiguity of business and accounting data is unlike engineering and
math. No matter the size of the database, there will be uncertainty and with the uncertainty the need for judgment, choice,
trust, and caution.
78 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
9. Conclusion
In the current age of Big Data, analysts need not only the skills to analyze data, but higher order skills such as the ability to
ask good questions and understand the limits of the analysis. We identified and classified a preliminary set of seven topics of
questions into a framework suitable for educational purposes. These questions are technology independent and emerge from
a deeper awareness of the assumptions that support Big Data analysis. We hope to initiate the conversation about these
questions by identifying and classifying seven questioning topics. We expect this list to grow and become more refined in
coming years, and that would be our suggestion for future research. Our goal is to help future accountants become informed
skeptics and recognize the opportunity and value of asking ‘‘good” questions in this Big Data era.
Appendix A
The supervision costs represent salary and benefit costs of the supervisors in charge of each product line.
Depreciation cost for machinery and equipment is charged to the product line on which the machinery is used.
The plant is leased. The lease rentals are charged to the product lines on the basis of square feet occupied.
Other fixed overhead costs are the cost of plant administration and are allocated arbitrarily by management decision.
These costs are unavoidable, regardless if the Baseball Equipment line is eliminated.
If the Baseball Equipment line is discontinued, the company will lose approximately 10% of its sales in each of the other
lines.
The equipment now used in the manufacture of Baseball Equipment is quite specialized. It has a current salvage value of
$105 and a remaining useful life of five years. This equipment cannot be used elsewhere in the company.
The plant space now occupied by the Baseball Equipment line could be closed off from the rest of the plant and rented for
$175 per year.
If the line is discontinued, the supervisor of the Baseball Equipment line will be released. In keeping with company policy
he would receive a one-time severance pay of $5.
Fixed selling and corporate administrative expenses are unavoidable.
Based upon the above additional data, Table 4 presents a reconciliation of the sign’s initial interpretation to its revised
interpretation.
Table 4
BBE sign reconciliation.
Implied annual profitability improvement of dropping the BBE line (loss avoidance) $ 124
Lost contribution margin from other lines (10%) (111)
Unavoidable costs arbitrarily allocated to BBE (204)
Sunk costs (depreciation) for BBE equipment (115)
Opportunity to sublease plant space occupied by BBE 175
Revised annual profitability impact of dropping the BBE line $(131)a
a
Ignores one time gain/loss on sale of equipment and severance payment.
References
Association for the Advancement of Collegiate Schools of Business [AACSB] (2014). AACSB International Accounting Accreditation Standard A7: Information
Technology Skills and Knowledge for Accounting Graduates: An Interpretation, White Paper issued by: AACSB International Committee on Accreditation
Policy AACSB International Accounting Accreditation Committee September.
Accounting Education Change Commission [AECC] (1990). Position statement no. one: Objectives of education for accountants. Issues in Accounting
Education, 5, 307–312 (1990, Fall).
American Accounting Association [AAA] (1986). Committee on the Future Structure, Content, and Scope of Accounting Education (the ‘‘Bedford
Committee”). Future accounting education: Preparing for the expanding profession. Issues in Accounting Education, 1, 168–195.
Anderson, C. (2008). The end of theory. Wired Magazine, 16(7). 16-07.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives.
978-0-8013-1903-7. Allyn and Bacon.
Babich, B. (1993). Philosophies of Science: Mach Duhem Bachelard, Research Resources. Paper 8.
Baril, C. P., Cunningham, B. M., Fordham, D. R., Gardner, R. L., & Wolcott, S. K. (1998). Critical thinking in the public accounting profession: Aptitudes and
attitudes. Journal of Accounting Education, 16(3), 381–406.
Behn, B. K., Ezzell, W. F., Murphy, L. A., Rayburn, J. D., Stith, M. T., & Strawser, J. R. (2012). The pathways commission on accounting higher education:
Charting a national strategy for the next generation of accountants. Issues in Accounting Education, 27(3), 595–600.
Beynon-Davies, P. (2009). Neolithic informatics: The nature of information. International Journal of Information Management, 29, 3–14.
Bisman, J. (2010). Postpositivism and accounting research: A (personal) primer on critical realism. Australasian Accounting Business & Finance Journal, 4(4),
3–25.
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. Harlow: Essex.
E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80 79
Boland, R. (1993). Accounting and the interpretive act. Accounting, Organizations and Society, 18(2), 125–146.
Boland, R. (1987). The in-formation of information systems. In K. Lyytinen, R. Boland, & R. Hirschheim (Eds.), Critical issues in information systems research.
Chichester: John Wiley and Sons.
Boland, R. J., Jr., & Schultze, U. (1996). From work to activity: Technology and the narrative of progress. In Information technology and changes in
organizational work (pp. 308–324). US: Springer.
Bollier, D. (2010). The promise and peril of Big Data. Washington D.C: The Aspen Institute.
Bonk, C. J., & Smith, G. S. (1998). Alternative instructional strategies for creative and critical thinking in the accounting curriculum. Journal of Accounting
Education, 16(2), 261–293.
Boyd, D., & Crawford (2011). Six provocations for Big Data. In Oxford internet institutes’ A decade in internet time symposium on the dynamics of the internet and
society. .
Chan, D. Y., & Kogan, A. (2016). Data analytics: Introduction to using analytics in auditing. Journal of Emerging Technologies in Accounting.
Chandler, D. (2007). Semiotics: The basics. Routledge.
Checkland, P., & Holwell, S. (1998). Information, systems, and information systems. New York: John Wiley and Sons.
Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188.
Coyne, J. G., Coyne, E. M., & Walker, K. B. (2016). A model to update accounting curricula for emerging technologies. Journal of Emerging Technologies in
Accounting, 13(1), 161–169.
Dretske, F. (1981). Knowledge and the flow of information. Cambridge, MA: MIT Press.
Ducheyne, S. (2008). JS Mill’s Canons of Induction: From true causes to provisional ones. History and Philosophy of Logic, 29(4), 361–376.
EY Reporting Issue 09. (2015). How big data and analytics are transforming the audit, April <http://www.ey.com/gl/en/services/assurance/ey-reporting-
issue-9-how-big-data-and-analytics-are-transforming-the-audit> (December 10, 2016).
Finley, W., & Waymire, T. R. (2013). Thinking practice: Iteration, peer review, and policy analysis in a governmental accounting class. Journal of Accounting
Education, 31(3), 333–349.
Floridi, L. (2005). Is information meaningful data. Philosophy and Phenomenological Research, 70(2), 351–370.
Frankfort-Nachmias, C., & Nachmias, D. (2007). Study guide for research methods in the social sciences. Macmillan.
Gantz, J., & Reinsel, D. (2012). The digital universe in 2020: Big Data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze the
Future.
Griffin, P. A., & Wright, A. M. (2015). Commentaries on Big Data’s importance for accounting and auditing. Accounting Horizons, 29(2), 377–379.
Hammond, J. S., Keeney, R. L., & Raiffa, H. (1998). The hidden traps in decision making. Harvard Business Review, 76(5), 47–58.
Hong, Y. Y., Morris, M. W., Chiu, C. Y., & Benet-Martinez, V. (2000). Multicultural minds: A dynamic constructivist approach to culture and cognition.
American Psychologist, 55(7), 709–715.
Huff, D. (1993). How to lie with statistics. WW Norton & Company.
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 263–291.
Kaisler, S., Armour, F., Espinosa, J., & Money, W. (2013). Big Data: Issues and challenges moving forward. In Proceedings of the 46th Hawaii international
conference on systems science (pp. 995–1004).
Kaisler, S., Mondy, W., & Coen, S. (2012). A decision framework for cloud computing. In Proceedings of the 45th Hawaii international conference on system
sciences, grand Wailea, Maui, HI, January 4–7. .
Kelley, H. H. (1967). Attribution theory in social psychology. In D. Levine (Ed.), Nebraska symposium on motivation (pp. 192–240). University of Nebraska
Press.
Kimmel, P. (1995). A framework for incorporating critical thinking into accounting education. Journal of Accounting Education, 13(3), 299–318.
Krahel, J. P., & Titera, W. R. (2015). Consequences of big data and formalization on accounting and auditing standards. Accounting Horizons, 29(2), 409–422.
Krumholz, H. M. (2014). Big data and new knowledge in medicine: The thinking, training, and tools needed for a learning health system. Health Affairs, 33(7),
1163–1170.
Kurfiss, J. G. (1988). Critical Thinking: Theory, Research, Practice, and Possibilities. ASHE-ERIC Higher Education Report No. 2, 1988. ASHE-ERIC Higher Education
Reports, The George Washington University, One Dupont Circle, Suite 630, Dept. RC, Washington, DC 20036–1183.
Laney, D. (2001). 3D data management: Controlling data volume, velocity, and variety. Gartner File, 949.
Lawson, R. A., Blocher, E. J., Brewer, P. C., Morris, J. T., Stocks, K. D., Sorensen, J. E., & Wouters, M. J. (2015). Thoughts on competency integration in accounting
education. Issues in Accounting Education, 30(3), 149–171.
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of google Flu: Traps in Big Data analysis. Science, 343, 1203–1205.
Lycett, M. (2013). ‘Datafication’: Making sense of (big) data in a complex world. European Journal of Information Systems, 22(4), 381–386.
MacKay, D. (1969). Information, mechanism, and meaning. Cambridge, MA: MIT Press.
McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J., & Barton, D. (2012). Big data. The management revolution. Harvard Business Review, 90(10), 61–67.
McGoun, E. G., Bettner, M. S., & Coyne, M. P. (2007). Pedagogic metaphors and the nature of accounting signification. Critical Perspectives on Accounting, 18
(2), 213–230.
McKinney, E., Yoos, C., Green, S., & Heppard, K. (2016). The impact of big data on accounting: changing the relationship between producers and consumers of
accounting digital goods. Journal of Emerging Technology Applications (submitted for publication).
McKinney, E., & Yoos, C. (2010). Information about information: A taxonomy of views. MIS Quarterly, 329–344.
McPherson, D. N. (1996). Student-initiated religious expression in the public schools: The need for a wider opening in the schoolhouse gate. Creighton Legal
Review, 30, 393.
Merino, J., Caballero, I., Rivas, B., Serrano, M., & Piattini, M. (2015). A data quality in use model for Big Data. Future Generation Computer Systems.
Mill, J. S. (1867). A system of logic, ratiocinative and inductive, being a connected view of the principles of evidence and the methods of scientific investigation. New
York: Longmans, Green, and Co.
O’Leary, M. B., Wilson, J. M., & Metiu, A. (2014). Beyond being there: The symbolic role of communication and identification in perceptions of proximity to
geographically dispersed colleagues. MIS Quarterly, 38(4), 1219–1243.
Albrecht, W. S., & Sack, R. J. (2000). Accounting education: Charting the course through a perilous future (Vol. 16) Sarasota, FL: American Accounting
Association.
Peirce, C. (1907). Excerpt from ‘‘Pragmatism”, published under the title ‘‘A Survey of Pragmaticism”. In Charles Hartshorne & Paul Weiss Cambridge (Eds.).
The collected Papers of Charles Sanders Peirce (Vol. 5). MA: Harvard University Press.
Previts, G., & Merino, B. (1979). A history of accounting in America. New York, NY: John Wiley.
Quine, W. V. O. (1990). Three indeterminacies. In R. B. Barrett & R. F. Gibson (Eds.), Perspectives on Quine (pp. 1–16). Cambridge, MA: Blackwell.
Ritchey, T. (2005). Wicked problems: Structuring social messes with morphological analysis. Swedish Morphological Society. swemorph.com/wp.html.
Ritson, P.A. (2001). Social constructionism in three accounting journals: 1977–1998. Collected Papers of APIRA Adelaide 2001.
Schwenk, C., & Valacich, J. S. (1994). Effects of devil0 s advocacy and dialectical inquiry on individuals versus groups. Organizational Behavior and Human
Decision Processes, 59(2), 210–222.
Scriven & Paul (1987). Critical thinking as defined by the National Council for Excellence. In Critical thinking 8th annual international conference on critical
thinking and education reform, summer 1987. .
Shah, S., Horne, A., & Capellá, J. (2012). Good data won’t guarantee good decisions. Harvard Business Review, 90(4).
Shore, B. (1996). Culture in mind: Cognition, culture, and the problem of meaning. Demand: Oxford University Press.
Siegler, M. (2010). Eric Schmidt: Every 2 days we create as much information as we did up to 2003, TechCrunch <http://techcrunch.com/2010/08/04/
schmidt-data/>.
80 E. McKinney Jr et al. / Journal of Accounting Education 38 (2017) 63–80
Šķiltere, D., & Jesiļevska, S. (2014). Data quality evaluation in statistical data processing. Statistical Journal of the IAOS, 30(4), 425–430.
Stamper, R. (1985). Toward a theory of information—information: Mystical fluid or a subject for scientific enquiry? The Computer Journal, 28(3), 195–199.
Stevens, J. P. (2012). Applied multivariate statistics for the social sciences. Routledge.
Stonebraker, M., & Hong, J. (2012). Researchers’ Big Data Crisis: Understanding design and functionality. Communications of the ACM, 55(2), 10–11.
Sullivan, D. (2015). NoSQL for mere mortals. Addison-Wesley Professional.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.
Twain, M. (1906). The private life of Adam and Eve: Being extracts from their diaries. Harper.
Velleman, P. F., & Wilkinson, L. (1993). Nominal, ordinal, interval, and ratio typologies are misleading. The American Statistician, 47(1), 65–72.
Vigo, R. (2011). Representational information: A general notion and measure of information. Information Sciences, 181, 4847–4859.
Warren, J. D., Jr, Moffitt, K. C., & Byrnes, P. (2015). How Big Data will change accounting. Accounting Horizons, 29(2), 397–407.
Yoon, K., Hoogduin, L., & Zhang, L. (2015). Big Data as complementary audit evidence. Accounting Horizons, 29(2), 431–438.
Zoglauer, T. (1996). Can information be naturalized? In K. Kornwachs & K. Jacoby (Eds.), Information: New questions to a multidisciplinary concept
(pp. 187–207). Berlin: Akademie-Verlag.