Next Article in Journal
Analysis of Users and Uses of DEMs in Spain
Previous Article in Journal
Virtual Geographic Simulation of Light Distribution within Three-Dimensional Plant Canopy Models
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJGI
Volume 6
Issue 12
10.3390/ijgi6120407
ijgi-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Hypothesis Platform: An Online Tool for Experimental Research into Work with Maps and Behavior in Electronic Environments

by
Čeněk Šašinka
1,*,
Kamil Morong
1 and
Zdeněk Stachoň
2,*
1
Department of Psychology, Centre for Experimental Psychology and Cognitive Sciences, Faculty of Arts, Masaryk University, 60200 Brno, Czech Republic
2
Department of Geography, Centre for Experimental Psychology and Cognitive Sciences, Faculty of Science, Masaryk University, 61137 Brno, Czech Republic
*
Authors to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2017, 6(12), 407; https://doi.org/10.3390/ijgi6120407
Submission received: 4 November 2017 / Revised: 6 December 2017 / Accepted: 11 December 2017 / Published: 20 December 2017
Browse Figures
Versions Notes

Abstract

:
The article presents a testing platform named Hypothesis. The software was developed primarily for the purposes of experimental research in cartography and psychological diagnostics. Hypothesis is an event-logger application which can be used for the recording of events and their real-time processing, if needed. The platform allows for the application of Computerized Adaptive Testing. The modularity of the platform makes it possible to integrate various Processing.js-based applications for creation and presentation of rich graphic material, interactive animations, and tasks involving manipulation with 3D objects. The Manager Module allows not only the administration of user accounts and tests but also serves as a data export tool. Raw data is exported from the central database in text format and then converted in the selection module into a format suitable for statistical analysis. The platform has many functions e.g., the creation and administration of tasks with real-time interaction between several participants (“multi-player function”) and those where a single user completes several tests simultaneously (“multi-task function”). The platform may be useful e.g., for research in experimental economics or for studies involving collaborative tasks. In addition, connection of the platform to an eye-tracking system is also possible.

1. Introduction

Research on user aspects of cartographic visualization can be traced back to the middle of the twentieth century. Contemporary technological developments enabled the extension of existing methods of spatial data presentation, such as contextual visualization [1], interactive (stereoscopic) 3D visualization [2,3,4], audiovisual communication [5], augmented reality [6] etc. Thus, there arises a need for the establishment of new research tools. The need for a distributed web-based approach is mentioned e.g., by Robinson [7]. There are several strategies on how to handle the issue. Most of the studies use an environment designed for a particular study. Robinson [7] demonstrated the usage of the e-Delphi platform and the e-Symbology Portal. This approach brings increased demands on software development for each performed study. Other studies usually use general software tools such as WebEx, Microsoft PowerPoint, Microsoft NetMeeting, etc. The advantage of the aforementioned general software is the general availability without the need for further application development. The main disadvantage consists of the limited possibility to customize the tools for a particular study. As there was no tool specifically designed for the large variety of usability testing issues in cartography, we decided to start development in this area. As many authors proved validity and profitability of remote usability testing [8], we began development in this direction.
The character of the newly designed platform was determined by cooperation between psychologists and cartographers. In the course of a research project entitled “Dynamic Geovisualization in Crisis Management” [9], a need arose to develop a research tool which would record the behavior of research subjects when working with cartographic materials (including interactive ones) while making it possible to create and administer psychological tests. Interactive electronic maps usually involve web browsing [10,11], so the new platform was to be designed as a web-based application. Based on a set of specifications, a client-server Multivariate Testing Programme (MuTeP) was developed which made it possible to perform varying map reading-related operations, including clicking on objects, route plotting, and others [12,13,14,15,16,17]. The same functionality was used to create psychological performance tests and tasks (e.g., adaptation of the framed-line test [18] and Embedded Figures Test [19]. However, the MuTeP application showed principal limitations in several aspects, one being the absence of adaptive testing principles. Based on the experience with the MuTeP application, a completely new platform, Hypothesis, was developed to overcome the limitations of the previous software [20]. In defining the architecture of the new platform, emphasis was placed on the range and variability of its functionality, and its flexibility when creating test tasks and batteries.

1.1. Application in the Field of Cognitive Cartography

Cartographic visualizations can be viewed as complex visual stimuli [21,22], where content and form interact. When creating test batteries, every task of the same tape is related to a different territory, and so different “correct answers” need to be defined. In Hypothesis, the necessary level of flexibility was achieved by the unique connection of slide templates with related slide contents (see Section 2.1), which makes it possible to perform all the necessary modifications of stimuli and correct answers. The varying activities and operations [23,24] performed by research subjects may range from simple visual searches, clicking on target objects [25], and sorting of objects of a single category [26] to more elaborate operations which include optimal route planning, terrain passability investigation [27,28], and even highly complex operations, such as those involving crisis management [29,30,31], agriculture-related decision-making [32], and crime analysis [33]. The unique design of Hypothesis makes it possible to conduct not only “molecular-level” experiments, but also studies focusing on “molar behavior” [34]. The aim of the former is to study low-level cognitive processes; they include visual search or memory tasks, where the speed, accuracy, and precision of the participant’s solution are analyzed. Molar-behavior studies, on the other hand, focus on high-level cognitive processes, investigating the strategies employed in the process of task solving. The analyzed behavior then includes a sequence of map-related operations (e.g., frequency of legend consulting, zooming, map shifting, switching between the varying map layers, selection of target areas, and their interconnection with optimal routes.

1.2. Application in the Field of Psychodiagnostics

Its wide functionality, central database, and well worked-out management module, along with the possibility to present rich and interactive visual stimulus, makes the Hypothesis platform suitable for psychological diagnostics as well. The platform allows for the creation (or adaptation) of a range of psychological performance tests which used to be available only in the paper-and-pencil version [35,36]. The platform contains expression evaluators so that the operations performed by participants can be evaluated in real time (while the participants are completing the tests). The table of results then contains not only raw data, but also the calculated scores and indices. The Hypothesis platform has already been used for the creation or adaptation of several psychological tests, including the D2 Test of Attention [37], Trail-making Test [38], and Grammatical Transformation Test [39], Perspective Taking Test or Mental Rotation Test [40,41].

2. Hypothesis: Characteristics and Architecture

The Hypothesis platform consists of a Module Manager and a Database. The Database, which contains whole Packs (Test Batteries) as well as the participants’ results, is located on the central server. The participants and database administrators access the Management Module (Figure 1) through a web browser. After log-in, the Management Module interface offers several functional modes. The participants can access only the Pack Menu (list of test batteries). Depending on the setting, the tests can be run in either the “Basic” mode (full screen window of a standard web browser) or in the “Controlled” mode (SWT browser). Administrators (managers or super-users) can, in addition to the list of tests, access the Administrator Account which makes it possible to control and manage all the user accounts. Also, the Module Manager contains an Export Module for the export of raw data (from the table of results) in .xlsx format. For the purpose of raw-data post-processing, a Selection Application was developed. When creating new tests, administrators also make use of the Slide Editor, where the individual slides can be displayed and edited. The slides can be edited using a web browser, while tests as such, are created in the Database.
The use and development of the Hypothesis platform are expected to be based on two principles: sharing and accumulation. Effective accumulation requires that the database is located in a centrally accessible server. “Sharing” involves not only utilization of already created Packs or their parts, but also the possibility to take part in their creation. The web-based design of the platform allows the publication of research studies along with the original stimulus (i.e., the tests used in the experiments) and experiment design. In addition, raw data can be made available for the research community. The process of “accumulation” can be done in two ways. First, varying tests and/or individual tasks can be created and then made accessible for future use (for instance, psychodiagnostic tests), whether in their original or partially modified form. The second method of “accumulation” consists of gradual implementation of new functionalities necessary for the creation of new tests. The newly created functionalities are then made available in the form of Slide Templates. Gradual accumulation of slide templates broadens the usability potential of the platform.

2.1. Hierarchical Structure of Packs

The highest-level unit of the hierarchy is a Pack (basically, a test) which is further divided into lower-level units. Each Pack consists of a set of slides, usually linearly arranged, meaning that they are presented to the participant in a fixed order. The option of random, tree or cyclic ordering of slides is also available (see Section 2.2). The slides are arbitrarily clustered into Tasks (usually based on the thematic link). Typically, the administrator includes several Tasks in a Pack, for instance: Task 1. Introduction and personal information; Task 2. Instructions; Task 3. Training task; Task 4. Test tasks; Task 5. Feedback and Conclusion. The clustering of slides into sections allows for simple and effective utilization of the thematic clusters in later tests. The individual levels of the Pack hierarchy are defined in the Database (Figure 2).
The basic unit of each Pack is the Slide. There are two components in each Slide: Slide Template and Slide Content (Figure 3). Each Slide Template should contain all the details concerning the structure of the relevant Slide; most importantly, it should include information on the visual arrangement of components, dialog windows, and functional logic of the Slide. Slide Templates (and Slides) are stored in the Database and are available to be used (whether in their original or customized form) by other research groups and in different experiments. The “functional logic” of a Slide concerns the response by the system to the operations performed by participants. The response is ensured by various control tools (for example, zoom, map shifting, point (polygon, broken line etc.) plotting, dialog window, timer), means of control (e.g., clicking, pressing of a button), definitions of actions related to the above events, definitions of variables used in each slide, definitions of any counters and, finally, definitions of slide-related output values saved into the Database. Each Slide Content is linked to the relevant Slide Template and determines the content that is to be presented in each Slide. For instance, if a Slide Template contains information about buttons (including their format), a picture and a text field, the Slide Content defines what buttons and picture are to be shown, what text is to be presented in the text field, and so on.
Tasks are linked to Packs indirectly, through Branches; they are linearly arranged within these Branches and presented to the participants in a fixed order. The purpose of a Branch is to allow for the effective creation of Packs for Computerized Adaptive Testing. In its simplest, linear form, a Test comprises a single Branch with a given number of Tasks (Figure 4a). More complex Packs have a tree structure, with participants going through the branches based on their current performance. Each Branch can be cyclically repeated until the participant achieves the required level of performance (e.g., percentage of correct answers in the training task; Figure 4b).

2.2. Computerized Adaptive Testing

The Hypothesis platform makes it possible to conduct Computerized Adaptive Testing (CAT) [43,44] at several levels of the hierarchy: slide level, task level, and branch level. At the slide level, events (e.g., mouse clicking) are paired with particular actions, with variables being assigned specific values which are subject to arithmetic and logic operations. Available algorithms include branching algorithms (IF-THEN-ELSE, SWITCH) and loops (WHILE), which are used to control slide-related actions. For the purposes of slide-level CAT optimization, a specialized slide was created (Image-sequence Layer; see Section 3.8). At the task level, each slide in a sequence of slides can be chosen based on the output related to the previous slide as well as on a set of pre-defined rules. The rules determining the course of each Task are a part of the Task table in the Database. A special way of ordering slides is randomization. In other words, there are three possible ways of slide ordering: linear, randomized, and adaptive. Adaptive ordering is available for the branch level as well, where each branch is chosen at the end of the previous branch.

2.3. Multi-Player/Task Tests

A significant aspect of the platform is the multi-player/task mode. It allows for real-time participation of two or more users in solving a single task (asynchronous testing); alternatively, one user may work on two different but interconnected tasks (synchronous testing).

2.3.1. Multi-Task Mode (Synchronous Testing)

Synchronous testing involves master and slave Packs. The Multi-task mode was developed for experiments requiring the participant to complete several tasks simultaneously, or to attend to several screens at the same time. The Multi-task mode was tested using an adaptation of the Peripheral Perception Test (PP-R Periphere Wahrnehmung-R) developed by Schuhfried [45]. The test is a part of the Vienna Test System (Germ. das Wiener Testsystem) and requires specialized hardware [46]. In the pilot experiment, whose purpose was to verify the synchronous testing functionality, the participants were asked to simultaneously complete a primary task (by clicking on target objects on the screen) and secondary tasks on peripheral screens using keyboard keys (Figure 5).

2.3.2. Multi-Player Mode (Asynchronous Testing)

The Multi-player Mode was originally intended for research in the field of geography, online collaboration respectively [47,48]. The mode makes it possible for several users to participate in solving a particular task. All the tests involved are controlled independently by the respective users; however, pauses in synchronicity may occur where an event requires a particular operation to be performed. While all participants are working on the same task, it is the administrator’s choice to define the conditions determining the course of testing, including, for instance, which participant has the first turn, in what order a picture is shown to the participants and so on. The Multi-player Mode has already been utilized in a series of adaptations of experiments related to social psychology (Figure 6) and behavioral economics [49,50].

2.4. Module Manager

The Manager Module represents an interface through which the software is accessed. Three user roles are defined: User, Manager and Superuser. “User” is at the lowest access level and can only select and start the tests made available by higher-level users. Superuser and Manager can access the Administrator Account, Export Module, and Slide Editor. Login is done via entering a user name and password. Unlogged users can enter the Module Manager as Guests, in which case only the free Packs are available.

2.4.1. Administrator Account

In the Administrator Account, each user can be provided with a name, password and access level (user role). The Superuser (highest-level user role) has unlimited administration rights with respect to all the other users, tests, and results. A Manager has administration rights over his/her group, within which s/he can create user accounts, allot Packs, and access all the results obtained by members of his/her group. S/he can also use the Slide Editor. The User role is to be assigned to research participants, whose rights are limited to starting a test Pack. Expiration date and manner of starting each Pack are specified by the Manager or Superuser (Figure 7). The Administrator Account allows for mass creation and editing of multiple user accounts; also, it makes it possible to export lists of user accounts (along with all the necessary details, including e.g., automatically generated passwords) in .xlsx format.

2.4.2. Slide Editor

Slide Editor is a part of the Module Manager functionality at the Superuser and Manager levels. In order to edit a slide using the editor, the user needs to copy its XML code (related to Content and Template) into corresponding windows of the Module Manager. The given slide then can be opened and edited. Verified XML codes are recorded into the Database.

2.4.3. Testing Modes and Pack Menu

In Hypothesis, there are several options for starting and running a test; of these, the most suitable one is chosen by the administrator. For the purpose of data collection where no control over experimental conditions is required (e.g., collection of data through online questionnaires), the test can be run using a standard web browser. The test opens in a pop-up, full-screen window; alternatively, a new browser window can be opened by clicking on a HASH link. The administrator can choose whether a user will be allowed to open a particular test in the controlled (featured) mode, in the standard (legacy) mode, or whether they will be allowed to choose (Figure 8). The controlled mode is realized through a browser based on SWT components. When opened, the SWT browser [51] requires installation of Java Runtime Environment. In the controlled mode, nearly all standard browser functions are disabled that could disrupt the highly controlled environment of the experiment. The SWT browser will open the test in a full-screen mode and will prevent the user from closing the application using standard means (Alt + F4 or Esc key); page refreshing and context menu are disabled as well. As Java gradually ceases to be supported in web browsers, an alternative way of running a controlled mode is currently being tested (see Section 6).

2.4.4. Export Module

In the Hypothesis platform, operations and events related to the course of testing are saved in the Database (table of Results). Key target variables are defined by the author (maker) of the test, who can also define the rules for their clustering and evaluation. The pre-defined values are then saved as special variables called Slide Outputs. The Export Module (Figure 9) is a part of the Module Manager; here, test Packs to be exported are selected (based on, for instance, the time of administration). The exported file is in .xslx format and contains raw data for further processing.

2.4.5. Selection Application

Effective selection (filtration) of variables from the raw data file is done in the selection application created in Visual Basic. The application searches through the data file, selecting values or chain parts in accordance with pre-defined rules. It always contains the following triplet:
  • List of raw data;
  • List of pre-defined variables, whose values are to be selected;
  • List of target values.

3. Slide Functionality

An important characteristic of the Hypothesis platform is modularity, which allows for continual development and broadening of the functionality of the software. The platform can be viewed as a high-performance core consisting of a content manager, processor for slide creation, and event logger with data store. An Open API makes it possible to create external modules and/or functions. The default configuration contains a set of basic components, including screen segmentation (vertical and horizontal), form features (e.g., TextField, ComboBox, SelectPanel), and also Button, Image, Panel, Label, and components for Audio and Video. Specific functionality is represented by a Dialog Window and Timer. Other components are a part of two separately supplied modules: Maps (for interactive maps) and Processing (for interactive animations created in the Processing language). See Supplementary Materials for examples of functionality.

3.1. Questionnaires

Questionnaires are commonly included in psychological research experiments. They may range from simple, personal information forms to Likert-type questionnaires. For the purpose of questionnaire creation, the software offers text fields, dropdown lists, scales, and others, with the possibility of intermediate validation of input values, which can be used to check whether obligatory information has been entered.

3.2. Visual Stimuli

Typically, visual stimuli are in the form of images. All standard raster graphics formats are supported (PNG, JPEG, GIF and similar ones). It is recommended to choose the format based on the type of content and with a view to minimum size of the file. For instance, GIF and PNG are suitable for schematic images with few colors, while JPEG should be used for photographs. The BMP file format is not recommended because it uses no data compression which results in enormously large files. When creating a slide with an image, longer loading time is to be expected, depending primarily on the file size and internet connection speed. Web browsers usually load images from cache, which makes loading of previously displayed images (identical image url) much faster. Different loading times can be compensated for by temporary masking of the images while they are loading. The Image component ensures recording of mouse clicks, which are provided with pixel coordinates.

3.3. Slide Control

Interaction with slide content can be realized through buttons or a button panel, e.g., for choosing between several options; alternatively, keyboard keys can be used (alphanumeric, arrow, or functional). The course of the test can also be controlled externally by Timer, with an optional display of time left. The timer can be stopped at any time and/or repeatedly activated. The most frequently used timer-related event is a timer end; however, timer start can be utilized as well. Timer can also be used to indicate that a pre-defined time interval has passed (event update). Button clicking, key pressing, and timer events are usually linked to slide-level actions.

3.4. Control of User Actions

At the slide level, various user actions can be defined which represent independent subroutines with specific names. Actions consist of evaluable expressions; these may be logical, mathematical, object-variable related, or branching expression-dependent (IF-THEN-ELSE, SWITCH-CASE). They are used to manage events, including, for instance, timer stop, key pressing, and clicking on a button/picture. Importantly, actions can call other actions, which is especially useful when branching is used, or when a more complex action is compiled and called by the individual parts. The advantage of actions is that they prevent the need to use an identical code several times; instead, a single action is called. Also, each action triggering is saved in the table of events. It follows from the above that using actions leads to higher transparency and comprehensibility of the expression of a functional algorithm related to a particular slide.

3.5. Dialog Window

The Dialog Window function may be used in order to repeatedly display Help while solving a task. The window appears after the calling of an appropriate function, which can be done e.g., by button clicking or any other pre-defined action. Closing is done by clicking on the cross button. Both the start and end of an action are saved into the Database. Dialog Window can either be non-modal (while it is active you can still work with the slide) or modal, meaning that the main window is disabled. Modal windows can be used to distinguish between currently running activities of the participant. The participant can only continue with his/her work after closing the modal window.

3.6. Maps

The map-related component is a part of the external plug-in module. The main purpose of the component consists of the implementation of maps into experiments. We distinguish between two main map layers: Base Layer and Feature Layer (with vector graphics). Examples of base layers include interactive Web Map Service (WMS) maps provided by servers offering WMS. The WMS layer requires specification of a coordinate reference system (CRS) and a bounding box. Depending on the data provided by the server, the WMS layer can contain either a single type of information or a complete set of cartographic details. Interactive features include zooming and shifting of the bounding box. Each change sends a request for data update to the server.
The Feature Layer contains vector objects; these can either be part of the default setting or they can be created by the participant using one of the special vector graphics tools. Various parameters of vector objects can be set including: stroke line color/style and fill style and color. Transparent vector objects are suitable to be used as Areas of Interest and may be utilized to assess a participant’s actions. Another important map layer is one with a static image, called Image Layer. This type of layer can be used instead of the “Image” basic component because it contains both the options offered by the Vector Layer and tools for vector object creation. The map component makes it possible to gather data about clicking on an object and/or using tools for the creation of lines, sections, broken lines, polygons and other geometric shapes/objects. If a coordinate reference system is specified (especially in connection with the WMS layer), the collected data may contain not only pixel coordinates, but also real geographical coordinates.

3.7. Audio and Video

The Hypothesis platform has two basic multimedia components: Audio (typically for MP3 audio format) and Video, which is usually used for MP4 files. An advantage of the Video component involves the possibility to record mouse clicking in a way similar to a static image. In addition to coordinates, the current time of audio-visual elements is recorded as well.

3.8. Image-Sequence Layer

The Image-Sequence Layer component significantly broadens the applicability of the Hypothesis platform. A test typically consists of a sequence of slides, which requires some loading time to be allowed. Therefore, no fast sequences of visual stimuli (at the order of tens of millisecond) would be possible. The speed of presentation of visual stimuli is normally limited by the speed and reliability of client-server communication. In order to overcome the above problem, a special component was designed which allows pre-loading of the stimulus material into the cache of the browser so that tachistoscopic tests can be performed [52]. Here, a presented slide is inactive (masked) until all the content has been loaded. Another key characteristic of the component is slide programmability. An algorithm defining the course of a slide can be written which also allows for slide-level adaptive testing (see above).

3.9. Processing.js Component

Another external module which broadens the applicability of the software is called Processing. Its purpose consists of the utilization of applications created in the Processing language (i.e., its web implementation, Processing.js), a visual programming language featuring a user-friendly developmental interface suitable for those with only basic knowledge of programming. The language is intended for data visualization, interactive animation, videogames, and other graphic material [53,54]. Processing.js is the sister project of the Processing programming language; it has been designed for online environments. The Processing code is based on JavaScript and can be run by any HTML5-compatible browser [55]. The interface between the Hypothesis platform and Processing.js makes it possible to create and administer a range of highly interactive tasks with visually rich and complex environments. Due to two-way communication between the Processing application and the core of the Hypothesis platform, the course of a task running in Processing.js can be controlled; at the same time, key events are being saved (in a pre-defined form) into the Database of the Hypothesis platform. Using the so called “call-back method”, each event/action occurring in Processing.js can call an action defined within a slide, which will affect the course of the slide. Similarly, a slide can call the processing code and thus influence the running of the Processing application.

4. Technical Design

The platform has been developed using the modern technology of the dynamic web page. The core and user interface are built on the Vaadin 7 framework [56]; database operations are ensured by ORM Hibernate, and PostgreSQL version 9.1 (or higher) is used as the primary database system. The application architecture is three-layer; a client, server, and database (Figure 10). The client part is responsible for user interaction and its operation is provided by a standard web browser (thin client) or a special browser distributed in the application package—Hypothesis Browser. This browser is built on Standard Widget Toolkit components and ensures more strict conditions for running tests. The client layer communicates in the background with the server through the technology Ajax RPC (remote procedure call). The server layer is implemented as a servlet of the application server (e.g., Apache Tomcat) which is responsible for the client’s request handling and updating the user interface. The servlet then communicates with the database layer by methods of object mapping of entities through the Hibernate library. This library provides a unified interface for the connection to all commonly used database systems (PostgreSQL, MySQL, MS SQL, Oracle, etc.). Individual Packs (test batteries) are structurally stored in the database. After starting a test, a selected package is loaded from the database to the server application and a new test entity is created. For more details see [42].

Time Measurement Accuracy

In the development of the software, great attention was paid to time measurement precision. The disadvantage of a client-server solution (as opposed to desktop applications), especially if the system responses are controlled by the server (which is the case of Hypothesis), which can lead to time delays during client-server communication. Delays in the server-to-client direction may occur e.g., as a result of the loading of images from a distant store. Client-to-server communication leads to a delayed response of the system to user actions. For instance, when a mouse click is sent from client to server, a delay occurs both in saving the event of clicking and sending back the system response. Negligible as they may seem (they are typically in the order of milliseconds), the delays are not invariable. The variability in client-server communication thus represents “noise” resulting in measurement inaccuracies. In order to compensate for this noise, several solutions have been implemented. First, the time of event occurrence at the client is recorded in addition to the time the event was accepted by the server, with the difference between these two representing the immediate time delay. When the system is located in a local network the delay tends to be negligible; with an online system it is dependent on external factors. Time delays can also be eliminated by the use of specialized “Image-Sequence Layer” slides (see Section 3.8) which prevent the delays related to the loading of content of the slide. Slides using the Processing.js plug-in relegate most of the event control to the client; therefore, the whole task is controlled locally, with only key events being sent to the server (for more details see the Supplement). Implementation of Processing.js fully eliminates the limits of the client-server solution and makes it possible to create tasks with high time accuracy (e.g., when an eye-tracking system is used).

5. Combination with Eye-Tracking

Although primarily intended for extensive research, the Hypothesis platform has, from the very beginning, been designed to support a relatively new intensive research method: eye-tracking. The use of Ogama, an already existing open-source application [57,58,59], and subsequent development of an online version called HypOgama made it possible to perform experiments involving a remote eye-tracking system SMI RED250 mobile or The Eyetribe. The existing solution is based on ex-post synchronization of an eye-tracker with the Hypothesis [60]. The above solution is a part of a wider concept which includes a ScanGraph tool for explorative data analysis [61]. At present, another version is being developed which will allow unified connection and real-time interaction between the Hypothesis platform and an eye-tracker. The solution is based on WebSocket communication elements of the Vaadin framework and its functionality has been verified in practice.

6. Comparison to Similar Tools

Nowadays, a range of tools are available which can be used to conduct computer-based experimental research. The development of the Hypothesis platform was motivated by the fact that there are currently no tools that would make it possible to collect a real-time record of an individual’s work with interactive maps and of collaboration between multiple users. The software’s architecture is highly universal and offers a unique combination of functions, one which is not provided by any other tool currently available. The software obviously has its limitations, too, and some types of experiments may want more specialized tools. Some applications, such as Presentation [62] and Psychtoolbox [63,64] were designed for experiments where high levels of precision are required; they are used for instance in neurocognitive research. Another example is E-prime [65], a software application with a large user base, which makes it possible to gather data using Tobii and SMI eye-tracking systems. There are numerous other applications available which offer eye-tracking-based data collection, including, for instance, the Paradigm software application [66], which allows for experiments to be conducted using both desktop and mobile devices. Free software applications from the above category include OpenSesame [67] and PsychoPy [68]; the latter is available online. In implementing eye-tracking into research experiments, some applications go a step further than the above-mentioned ones, combining inventive data collection using an eye-tracker with effective data analysis. Examples include the “Experiment Builder” and “Data Viewer” [69], Experiment Center and BeGaze [70] and Ogama [57]; all of these are applications provided by eye-tracker manufacturers. The limitations of these applications are related to basic functions in test creation. For instance, the Experiment Center and Ogama do not allow for several Likert scales to be used within a single slide; also, they do not make it possible to collect data online. Examples of free psychology software applications for online data collection include PEBL [71], Tatool [72], Social Lab [73] and jsPsych [74]. The latter two web-based tools in particular are conceptually similar to the Hypothesis platform, which, however, offers significantly broader functionality, with an emphasis on the controlled nature of the experiment. Among the areas which can benefit from the advantages of the Hypothesis platform is the field of cartography, which has a long tradition of employing usability tests; the importance of usability studies for cartography was emphasized in the International Cartographic Association Research Agenda published in 2009 [75]. Yet, the field has long been lacking a suitable tool for conducting usability tests. The lack of tools for usability studies in cartography is mentioned in Nivala et al. [76]. Even recently published papers still use domain-unspecific software and tools [77] and self-developed tools created ad hoc [78]. To our knowledge, cartography at present has no universal experimental tool for usability tests; thus, researchers have to use the software and tools mentioned above. Unlike these, the Hypothesis platform makes it possible to collect data online and in a controlled environment, offering high temporal precision of measurements. It is also suitable for multitask/multiplayer experiments and for investigating a user’s behavior when they are working with interactive maps.

7. Test of Resources

During its development, the Hypothesis application was tested several times; also, feedback was requested from the researchers who used the platform when carrying out their experiments. In 2015, Z. Štěrba and J. Čeněk (both Masaryk University) performed an intercultural study consisting of a series of experiments in which 106 university students participated. The experiments were conducted at universities in the Czech Republic, China and Switzerland. The study used a set of tests involving different functionalities (including clicking, line drawing and Likert scales). When administering the tests, two relatively marginal problems were encountered. One of them consisted in different load times of the stimulus material, which was caused by varying quality of internet connection in different places; the application runs on a server located at Masaryk University in Brno, Czech Republic. A more substantial problem was related to the instability of the system during mass testing, when administering 10 to 15 (and more) tests simultaneously. There were several cases of the application “freezing” in mid-test during the first load test. A similar situation was encountered by Svatoňová and Kolejka [79] or Knedlová [80] and Helísková [81] while they were conducting research towards their theses. When a high number of tests were run simultaneously, the test shut down prematurely, leading to a loss of data. The tests were administered to 52 participants at the premises of Masaryk University. The functionalities used in the tests included multi-clicking (with feedback), line and area drawing with automatic evaluation, text fields, scroll-down windows, an image-sequence layer slide with a button bar and feedback, and keyboard control. Apart from the limitation in the number of tests that could be run simultaneously, no other significant problem was encountered during the testing. The cause of the problem was revealed to be related to “sessions”, which are automatically created upon starting a test but did not always close as expected. After the error was resolved, the application was used by other research groups, for instance by Kubíček et al. [82], who conducted a study concerning mental rotations where a maximum of 5 participants were tested simultaneously; Opach et al. [83] (individual testing of 25 participants). No problem related to the Hypothesis platform was reported by any of the above research groups. After the implementation of the newest version of the platform, another load test was carried out. The test consisted of logging into the Manager Module of the platform 16 times (8× in Firefox and 8× in Chrome) and launching 82 identical tests simultaneously. Each test contained 2 slides of the image-sequence-layer type. As was expected, no complications occurred during the second load test, with all the data collected being saved into the database (see Supplementary Materials).

8. An Experiment Illustrating the Usability of the Hypothesis Platform

8.1. Introduction

We are presenting a conducted experiment in order to document the wide range of functionality, usability and stability of the Hypothesis platform. Our colleague, Markéta Kukaňová, conducted a complex study which explored the effectiveness and efficiency of contextual cartographic visualizations [84]. She compared work with usual topographic map and work with a combination of topographic and transport maps. She conducted a simulation using contextual visualization techniques in the transport planning and also measured the cognitive abilities of users which may have positively influenced performance during the map reading process. Because of the above-mentioned reasons, she used the Hypothesis platform, which made the creation and administration of all the necessary tasks and tests possible.

8.2. Methods

The main goal of the empirical study was to compare two variants of spatial data representation. Throughout the entire experiment, the control group only worked with standard military topographic maps. Alternately, the experimental group worked with standard military topographic maps and with adapted visualization focused on the tasks connected with transportation. Both visualizations were informationally equivalent. The whole battery consisted of two cartographic tests and four psychological tests (identical for both groups; Figure 11).
The main principle of the contextual maps is that the visualization adapts to the specific activity of the user. During free map exploration or a visual search for target objects, the basic topographic map is used. When the type of activity is changes e.g., to transportation planning, visualization is switched to a specific transport map, where the transport infrastructure is highlighted and unnecessary information is suppressed (Figure 12).
Cartographic test 1 consisted of two subtests. The participants should search for target objects in the first subtest. The second subtest was similar to the first one but the participants were listening to an audio recording simultaneously, and were expected to react with a key press according to word cues. The correctness, visual search time, and reaction time to target words were measured. Alternately, the experimental group worked with topographic and transport maps. The level of cognitive load was explored in these types of tasks.
Cartographic test 2 consisted of four subsequent stages. Participants explored a map with multi-click marked target objects in the first stage. The reaction time and correctness was measured. In the second stage, participants localized a fire based on the instructions; afterwards they located the position of the fire truck. In the third stage, participants drew the optimal route between the fire and the fire truck. The control group always worked with a topographic map while the experimental group worked with a transport map during the third stage. The routes drawn were recorded to a database as were their reaction times. In the final and fourth stage, a blank map was presented. Participants were required to estimate the position of the objects which were previously presented in the first three stages. Accuracy of the marked objects was measured in pixels.
Every single test from psychological portion used different functionality. The Flagtest (measurement of selective attention) involved, among others, following functionality: multi-click, timer or feature layer. The Compound Figure Test (measurement of global/local precedence) used image sequence layer, real-time feedback, automatic evaluation of the assessment, button bar. In the Absolute Relative Test (measurement of holistic/analytic cognitive style) users estimated the size of previously seen objects with the help of drawn horizontal lines. Hypothesis calculated deviation (accuracy) in pixels. And finally, the Stroop Spatial Test (measurement of cognitive control) was based on image sequence layer, participants used a keyboard to react to the stimuli.
Totally 43 individuals (age was between 18 and 44; m = 26.6; men = 24, women = 19) participated in the study. There were 20 participant in control group and 23 in experimental group. A part of the research sample (n = 27) was tracked simultaneously with the eye-tracking system (The Eyetribe eyetracker and software OGAMA).

8.3. Results

After the data collection was finished, all data were exported to .xlsx files from the database of Hypothesis with the help of the export module. Afterwards, the selection application was used and the required variables were filtered and stored in a format enabling subsequent analysis. Data were analyzed in the IBM SPSS program and some analysis were also provided and visualized with the help of ArcGIS 10.2.2. No problems occurred during data collection that could be linked to the Hypothesis platform. However, some participants did not finish all the tests included in the test battery because of problems with the administrator’s PC and internet connectivity issues. Also, there was some data loss with those participants who were measured with the eye-tracking system. The OGAMA software showed instability during longer tests. Figure 13 shows an analysis of the routes drawn in cartographic experiment 1. Green digits refer to the experimental group (context transport visualization), blue refer to the control group (topographic map). Routes drawn by 36 participants were analyzed.

8.4. Discussion

The experiment conducted by Markéta Kukaňová clearly showed that the Hypothesis platform provides a range scale of functionality which enables the creation and administration of very different types of tasks. We are not aware of another research tool which would be able to provide all the necessary functionality for the realization of the above-described study. Without Hypothesis, a research study of this nature would have to use two or more different tools, and this of course would increase the costs of the research and make it more complicated. The Hypothesis platform also enabled the exportation and filtering of all the necessary variables in order to conduct the intended analysis. Figure 13 showed that even a research variable which has many degrees of freedom could be properly analyzed. These features offer a strong advantage over other research tools which are mostly able to present only very simple stimuli and collect simple responses such as a mouse click or a keypress. The most important thing should always be the robustness of the research tools. Hypothesis worked without a single problem, however, the general limitations of the web-based tools were detected. The stability of the internet connections played a crucial role in this case, too. An internet connection lost in even one single test may devalue the data from the affected participant for the entire study. For these reasons, it should be recommended that data be collected only under the highest quality conditions.

9. Conclusions

An original research tool, the Hypothesis platform, was described and its usability was presented in detail in the context of cognitive cartography and psychological testing. Its comparison to other research tools was presented, as was the testing and evaluation of the Hypothesis platform. One conducted experiment was chosen and described in more detail in order to make visible the range of functionality which was involved in one particular study. Other experiments were also referred to, some which were conducted by students for their diploma thesis, some by researchers who published their work in well-established journals. The usability of the current version of the Hypothesis platform was proven for research purposes and for educational purposes. However, the software is still being developed and new functionality is constantly incorporated in order to react to the development in the area of cognitive cartography. Current developmental efforts focus on 3D technology implementation (pilot experimental testing of 3D visualization using web technologies has already been made [85,86]), optimization for tablets or other devices with touch screens [87], interconnection with an eye-tracking system, and further development of a Firefox-based browser. The new application using Firefox was able to replace the original SWT-browser based solution in controlled data collection. Also, an entire graphical user interface is being developed which can be utilized in test battery creation, and which will allow access (and editing) through a standard web browser. The Hypothesis platform is now available for academic and other non-commercial purposes (under Masaryk University). However, special agreements may be negotiated with respect to commercial use or other specific purposes.

Supplementary Materials

The test data are available at: https://doi.org/10.6084/m9.figshare.4697674.v1; Installation of Hypothesis with examples of functionality for demonstrative purposes is available at (user name: isprs password: isprs): http://demo-hypothesis.phil.muni.cz; Tutorial “how to modify slides and work with the Slide Management”.

Acknowledgments

This research was funded by projects “Influence of cartographic visualization methods on the success of solving practical and educational spatial tasks” [grant number MUNI/M/0846/2015] and “Integrated research on environmental changes in the landscape sphere of Earth II” [grant number MUNI/A/1419/2016], both awarded by Masaryk University, Czech Republic. This research was supported by the research infrastructure HUME lab Experimental Humanities Lab, Faculty of Arts, Masaryk University too.

Author Contributions

Kamil Morong and Čeněk Šašinka designed the architecture of the software. Kamil Morong wrote the code of the software. Čeněk Šašinka and Zdeněk Stachoň designed and performed all empirical testing and analyzed the data. Čeněk Šašinka, Zdeněk Stachoň and Kamil Morong wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Mulíčková, E.; Šafr, G.; Staněk, K. Context map—A Tool for Cartography Support in Crisis Management. In Proceedings of the 3rd International Conference on Cartography and GIS, Nessebar, Bulgaria, 15–20 June 2010. [Google Scholar]
  2. Herman, L.; Popelka, S.; Hejlová, V. Eye-tracking Analysis of Interactive 3D Geovisualizations. J. Eye Mov. Res. 2017, 10, 1–15. [Google Scholar] [CrossRef]
  3. Špriňarová, K.; Juřík, V.; Šašinka, Č.; Herman, L.; Štěrba, Z.; Stachoň, Z.; Chmelík, J.; Kozlíková, B. Human-computer Interaction in Real 3D and Pseudo-3D Cartographic Visualization: A Comparative Study. In Cartography—Maps Connecting the World; Sluter, C.R., Cruz, C.B.M., de Menezes, P.M.L., Eds.; Springer International Publishing: Basel, Switzerland, 2015; pp. 59–73. [Google Scholar]
  4. Herbert, G.; Chen, X. A Comparison of Usefulness of 2D and 3D Representations of Urban Planning. Cartogr. Geogr. Inf. Sci. 2014, 42, 22–32. [Google Scholar] [CrossRef]
  5. Lammert-Siepmann, N.; Bestgen, A.-K.; Edler, D.; Kuchinke, L.; Dickmann, F. Audiovisual communication of object-names improves the spatial accuracy of recalled object-locations in topographic maps. PLoS ONE 2017, 12, e0186065. [Google Scholar] [CrossRef] [PubMed]
  6. Chmelařová, K.; Šašinka, Č.; Stachoň, Z. Visualization of Environment-related Information in Augmented Reality: Analysis of User Needs. In Advances in Cartography and GIScience; Peterson, M., Ed.; ICACI 2017, Lecture Notes in Geoinformation and Cartography; Springer: Washington, DC, USA, 2017; pp. 283–292. [Google Scholar]
  7. Robinson, A. Challenges and opportunities for web-based evaluation of the use of spatial technologies. In Proceedings of the ICA Conference, The Pennsylvania State University, University Park, PA, USA, 3–8 July 2011. [Google Scholar]
  8. Tullis, T.; Fleischman, S.; Mcnulty, M.; Cianchette, C.; Bergel, M. An empirical comparison of lab and remote usability testing of Web sites. In Proceedings of the Usability Professionnal Association Conference, Boston, MA, USA, 8–12 July 2002. [Google Scholar]
  9. Konečný, M.; Březinová, Š.; Drápela, M.V.; Friedmannová, L.; Herman, L.; Hübnerová, Z.; Kolář, M.; Kolejka, J.; Kozel, J.; Kubíček, P.; et al. Dynamická Geovizualizace V Krizovém Managementu, 1st ed.; Masarykova univerzita: Brno, Czech Republic, 2011. [Google Scholar]
  10. Evans, B.; Sabel, C.E. Open-Source web-based geographical information system for health exposure assessment. Int. J. Health Geogr. 2012, 11, 1–12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Řezník, T.; Horáková, B.; Szturc, R. Geographic Information for Command and Control Systems Demonstration of Emergency Support System. In Intelligent Systems for Crisis Management: Geo-Information for Disaster Management (GI4DM) Lecture Notes in Geoinformation and Cartography; Zlatanova, S., Peters, R., Dilo, A., Scholten, H., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 263–275. [Google Scholar] [CrossRef]
  12. Kubíček, P.; Šašinka, Č.; Stachoň, Z.; Štěrba, Z.; Apeltauer, J.; Urbánek, T. Cartographic Design and Usability of Visual Variables for Linear Features. Cartogr. J. 2016, 2016, 1–11. [Google Scholar] [CrossRef]
  13. Kubíček, P.; Šašinka, Č. Thematic Uncertainty Visualization Usability–Basic Methods Comparison. Ann. GIS 2011, 17, 253–263. [Google Scholar] [CrossRef]
  14. Štěrba, Z.; Šašinka, Č.; Stachoň, Z.; Kubíček, P.; Tamm, S. Mixed Research Design in Cartography: A Combination of Qualitative and Quantitative Approaches. Kartogr. Nachr. 2014, 64, 262–269. [Google Scholar]
  15. Kubíček, P.; Šašinka, Č.; Stachoň, Z. Vybrané kognitivní aspekty vizualizace polohové nejistoty v geografických datech. Geogra-Sb. České Geogra Spol. 2014, 119, 67–90. [Google Scholar]
  16. Konečný, M.; Kubíček, P.; Stachoň, Z.; Šašinka, Č. The usability of selected base maps for crises management: Users’ perspectives. Appl. Geomat. 2011, 3, 189–198. [Google Scholar] [CrossRef]
  17. Stachoň, Z.; Šašinka, Č.; Štěrba, Z.; Zbořil, J.; Březinová, Š.; Švancara, J. Influence of Graphic Design of Cartographic Symbols on Perception Structure. Kartogr. Nachr. 2013, 4, 216–220. [Google Scholar]
  18. Kitayama, S.; Duffy, S.; Kawamura, T.; Larsen, J.T. Perceiving an object and its context in different cultures: A cultural look at new look. Psychol. Sci. 2003, 14, 201–206. [Google Scholar] [CrossRef] [PubMed]
  19. Witkin, H.A.; Oltman, P.K.; Raskin, E.; Karp, S. A Manual for the Embedded Figures Test; Consulting Psychologists Press: Palo Alto, CA, USA, 1971. [Google Scholar]
  20. Morong, K.; Šašinka, Č. Hypothesis—online software platform for objective experimental testing. In Applying Principles of Cognitive Psychology in Practice: Conference Proceedings; Bartošová, K., Čerňák, M., Kukaňová, M., Šašinka, Č., Eds.; Masaryk University: Brno, Czech Republic, 2015; Volume 5, pp. 15–16. [Google Scholar]
  21. Thorndyke, P.W.; Stasz, C. Individual differences in procedures for knowledge acquisition from maps. Cogn. Psychol. 1980, 12, 137–175. [Google Scholar] [CrossRef]
  22. Edler, D.; Bestgen, A.-K.; Kuchinke, L.; Dickmann, F. Grids in Topographic Maps Reduce Distortions in the Recall of Learned Object Locations. PLoS ONE 2014. [Google Scholar] [CrossRef] [PubMed]
  23. Crampton, J.W. Interactivity types in geographic visualization. Cartogr. Geogr. Inf. Sci. 2002, 29, 85–98. [Google Scholar] [CrossRef]
  24. Roth, R.E. An empirically-derived taxonomy of interaction primitives for Interactive Cartography and Geovisualization. Trans. Vis. Comput. Graph. 2013, 19, 2356–2365. [Google Scholar] [CrossRef] [PubMed]
  25. Popelka, S.; Brychtová, A. Eye-tracking Study on Different Perception of 2D and 3D Terrain Visualization. Cartogr. J. 2013, 50, 240–246. [Google Scholar] [CrossRef]
  26. Brychtová, A.; Coltekin, A. An Empirical User Study for Measuring the Influence of Colour Distance and Font Size in Map Reading Using Eye Tracking. Cartogr. J. 2016, 53, 202–212. [Google Scholar] [CrossRef]
  27. Hofmann, A.; Hošková-Mayerová, Š.; Talhofer, V.; Kovařík, V. Creation of models for calculation of coefficients of terrain passability. Qual. Quant. 2015, 49, 1679–1691. [Google Scholar] [CrossRef]
  28. Devlin, A.S.; Bernstein, J. Interactive way-finding: map style and effectiveness. J. Environ. Psychol. 1997, 17, 99–110. [Google Scholar] [CrossRef]
  29. Vakalis, D.; Sarimveis, H.; Kiranoudis, C.T.; Alexandridis, A.; Bafas, G. A GIS based operational system for wildland fire crisis management II. System architecture and case studies. Appl. Math. Model. 2004, 28, 411–425. [Google Scholar] [CrossRef]
  30. Ilmavirta, A. The use of GIS-system in catastrophe and emergency management in Finnish municipalities. Comput. Environ. Urban Syst. 1995, 19, 171–178. [Google Scholar] [CrossRef]
  31. Konečný, M.; Friedmannová, L.; Stanek, K. An adaptive cartographic visualization for support of the crisis management. In CaGIS Publications—Autocarto 2006; CaGIS: Vancouver, WA, USA, 2006; pp. 100–105. [Google Scholar]
  32. Kubíček, P.; Kozel, J.; Štampach, R.; Lukas, V. Prototyping the visualization of geographic and sensor data for agriculture. Comput. Electron. Agric. 2013, 97, 83–91. [Google Scholar] [CrossRef]
  33. Roth, R.E.; Ross, K.S.; MacEachren, A.M. User-centered design for interactive maps: A case study in crime analysis. Int. J. Geo-Inf. 2015, 4, 262–301. [Google Scholar] [CrossRef]
  34. Baum, W.M. From molecular to molar: A paradigm shift in behavior analysis. J. Exp. Anal. Behav. 2002, 78, 95–116. [Google Scholar] [CrossRef] [PubMed]
  35. Mead, A.D.; Drasgow, F. Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychol. Bull. 1993, 114, 449–458. [Google Scholar] [CrossRef]
  36. Květon, P.; Jelínek, M.; Vobořil, D.; Klimusová, H. Computer-based tests: The impact of test design and problem of equivalency. Comput. Hum. Behav. 2007, 23, 32–51. [Google Scholar] [CrossRef]
  37. Ross, R.M. The D2 Test of Attention: An Examination of Age, Gender, and Cross-Cultural Indices; Argosy University: Chicago, IL, USA, 2005. [Google Scholar]
  38. Arnett, J.A.; Labovitz, S.S. Effect of physical layout in performance of the Trail Making Test. Psychol. Assess. 1995, 7, 220–221. [Google Scholar] [CrossRef]
  39. Baddeley, A.D. A 3 min reasoning test based on grammatical transformation. Psychon. Sci. 1968, 10, 341–342. [Google Scholar] [CrossRef]
  40. Kozhevnikov, M.; Hegarty, M. A dissociation between object-manipulation and perspective-taking spatial abilities. Mem. Cognit. 2001, 29, 745–756. [Google Scholar] [CrossRef] [PubMed]
  41. Hegarty, M.; Waller, D. A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence 2004, 32, 175–191. [Google Scholar] [CrossRef]
  42. Štěrba, Z.; Šašinka, Č.; Stachoň, Z.; Štampach, R.; Morong, K. Selected Issues of Experimental Testing in Cartography; Masaryk University: Brno, Czech Republic, 2015. [Google Scholar]
  43. Žitný, P.; Halama, P.; Jelínek, M.; Květon, P. Validity of cognitive ability tests–comparison of computerized adaptive testing with paper and pencil and computer-based forms of administrations. Stud. Psychol. 2012, 54, 181–194. [Google Scholar]
  44. Gershon, R.C. Computer Adaptive Testing. J. Appl. Meas. 2005, 6, 109–127. [Google Scholar] [PubMed]
  45. Karner, T.; Neuwirth, W. Die Bedeutung der peripheren Wahrnehmung in der verkehrspsychologischen Untersuchung. Psychol. Österreich 2001, 21, 183–186. [Google Scholar]
  46. Schuhfried, G. PP-R Periphere Wahrnehmung—R. Available online: https://www.schuhfried.at/test/PP-R (accessed on 20 October 2016).
  47. Krek, A.; Bortenschlager, M. Geo-collaboration and P2P Geographic Information Systems. Current Developments and Research Challenges. Collaborative Peer to Peer Information Systems (COPS06) Workshop-WETICE. Manchester, UK, 2006. Available online: http://www.sel.uniroma2.it/cops06/papers/COPS06-Krek.pdf (accessed on 20 October 2016).
  48. MacEachren, A.M.; Cai, G.; Sharma, R.; Rauschert, I.; Brewer, I.; Bolelli, L.; Shaparenko, B.; Fuhrmann, S.; Wang, H. Enabling collaborative geoinformation access and decision-making through a natural, multimodal interface. Int. J. Geogr. Inf. Sci. 2005, 19, 293–317. [Google Scholar] [CrossRef]
  49. Asch, S.E. Effects of Group Pressure on the Modification and Distortion of Judgments. In Groups, Leadership and Men; Guetzkow, H., Ed.; Carnegie Press: Pittsburgh, PA, USA, 1951; pp. 177–190. [Google Scholar]
  50. Ciampaglia, G.L.; Lozano, S.; Helbing, D. Power and Fairness in a Generalized Ultimatum Game. PLoS ONE 2014. [Google Scholar] [CrossRef] [PubMed]
  51. Cornu, C.H. SWT Browser: Viewing HTML Pages with SWT Browser Widget. 26 August 2004. Available online: https://eclipse.org/articles/Article-SWT-browser-widget/browser.html (accessed on 25 October 2016).
  52. Godnig, E.C. The Tachistoscope: Its History and Uses. J. Behav. Opt. 2003, 14, 39–42. [Google Scholar]
  53. Shiffman, D. Learning Processing: A Beginner’s Guide to Programming Images, Animation, and Interaction, 1st ed.; Morgan Kaufmann: Burlington, MA, USA, 2008. [Google Scholar]
  54. Reas, C.; Fry, B. Getting Started with Processing, 1st ed.; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2010. [Google Scholar]
  55. A Port of the Processing Visualization Language. Available online: http://processingjs.org/ (accessed on 25 October 2016).
  56. Vaadin Ltd. 2016. Available online: https://vaadin.com/home (accessed on 25 October 2016).
  57. Voßkühler, A.; Nordmeier, V.; Kuchinke, L.; Jacobs, A.M. OGAMA–OpenGazeAndMouseAnalyzer: Open source software designed to analyze eye and mouse movements in slideshow study designs. Behav. Res. Method. 2008, 40, 1150–1162. [Google Scholar] [CrossRef] [PubMed]
  58. Voßkühler, A. OGAMA Description (for Version 2.5). A Software to Record, Analyze and Visualize Gaze and Mouse Movements in Screen Based environments. 2009. Available online: http://www.ogama.net/sites/default/files/pdf/OGAMA-DescriptionV25.pdf (accessed on 30 October 2016).
  59. Voßkühler, A. OGAMA (OpenGazeAndMouseAnalyzer): An Open Source Software Designed to Analyze Eye and Mouse Movements in Slideshow Study Designs. 16 May 2015. Available online: www.ogama.net (accessed on 30 October 2016).
  60. Popelka, S.; Stachoň, Z.; Šašinka, Č.; Doležalová, J. Eyetribe Tracker Data Accuracy Evaluation and Its Interconnection with Hypothesis Software for Cartographic Purposes. Comput. Intell. Neurosci. 2016, 1–14. [Google Scholar] [CrossRef] [PubMed]
  61. Doležalová, J.; Popelka, S. ScanGraph: A novel scanpath comparison method using graph cliques visualization. J. Eye Mov. Res. 2016, 9, 1–13. [Google Scholar] [CrossRef]
  62. Neuro behavioral systems, Inc. Available online: www.neurobs.com (accessed on 24 February 2017).
  63. Cornelissen, F.W.; Peters, E.M.; Palmer, J. The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox. Behav. Res. Method. Instrum. Comput. 2002, 34, 613–671. [Google Scholar] [CrossRef]
  64. Psychtoolbox-3. Available online: http://psychtoolbox.org (accessed on 24 February 2017).
  65. Psychology Software Tools, Inc. Available online: www.pstnet.com (accessed on 24 February 2017).
  66. Paradigm. Available online: http://www.paradigmexperiments.com (accessed on 24 February 2017).
  67. Mathôt, S.; Schreij, D.; Theeuwes, J. OpenSesame: An open-source, graphical experiment builder for the social sciences. Behav. Res. Method. 2012, 44, 314–324. [Google Scholar] [CrossRef] [PubMed]
  68. Pliatsikas, C.; Johnstone, T.; Marinis, T. FMRI evidence for the involvement of the procedural memory system in morphological processing of a second language. PLoS ONE 2014, 9, e97298. [Google Scholar] [CrossRef] [PubMed]
  69. SR Research: Fast, Accurate, Reliable Eye Tracking. Available online: http://www.sr-research.com (accessed on 24 February 2017).
  70. SMI SensoMotoric Instruments. Available online: www.smivision.com (accessed on 24 February 2017).
  71. PEBL. The Psychology Experiment Building Language. Available online: http://pebl.sourceforge.net (accessed on 24 February 2017).
  72. Von Bastian, C.C.; Locher, A.; Ruflin, M. Tatool: A Java-Based Open-Source Programming Framework for Psychological Studies. Behav. Res. Methods 2013, 45, 108–115. [Google Scholar] [CrossRef] [PubMed]
  73. Garaizar, P.; Reips, U.D. Build your own social network laboratory with Social Lab: A tool for research in social media. Behav. Res. Methods 2014, 46, 430–438. [Google Scholar] [CrossRef] [PubMed]
  74. De Leeuw, J.R. Jspsych: A JavaScript library for creating behavioral experiments in a web browser. Behav. Res. Methods 2015, 47, 1–12. [Google Scholar] [CrossRef] [PubMed]
  75. Virrantaus, K.; Fairbairn, D.; Kraak, M. ICA research agenda on cartography and GIScience. Cartogr. Geogr. Inf. Sci. 2009, 36, 209–222. [Google Scholar] [CrossRef]
  76. Nivala, A.-M.; Sarjakoski, L.T.; Sarjakoski, T. Usability methods’ familiarity among map application developers. Int. J. Hum.-Comput. Stud. 2007, 65, 784–795. [Google Scholar] [CrossRef]
  77. Ooms, K.; Coltekin, A.; De Maeyer, P.; Dupont, L.; Fabrikant, S.; Incoul, A.; Kuhn, M.; Slabbick, H.; Vansteenkiste, P.; Van der Haegen, L. Combining user logging with eye tracking for interactive and dynamic applications. Behav. Res. Methods 2015, 47, 977–993. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  78. Demšar, U. Investigating visual exploration of geospatial data: An exploratory usability experiment for visual data mining. Comput. Environ. Urban Syst. 2007, 31, 551–571. [Google Scholar] [CrossRef]
  79. Svatoňová, H.; Kolejka, J. Comparative Research of Visual Interpretation of Aerial Images and Topographic Maps for Unskilled Users: Searching for Objects Important for Decision-Making in Crisis Situations. ISPRS Int. J. Geo-Inf. 2017, 6, 231. [Google Scholar] [CrossRef]
  80. Helisková, M. Možnosti A Limity Online Počítačové Diagnostiky. The Capabilities and the liMits of Online PC Diagnostics. Master’s Thesis, Masaryk University, Brno, Czech Republic, December 2016. [Google Scholar]
  81. Knedlová, P. Kognitivní styl a jeho vliv na čtení mapy. The Influence of Cognitive Style on Map Reading. Master’s Thesis, Masaryk University, Brno, Czech Republic, 2016. [Google Scholar]
  82. Kubíček, P.; Šašinka, Č.; Stachoň, Z.; Herman, L.; Juřík, V.; Urbánek, T.; Chmelík, J. Identification of altitude profiles in 3D geovisualizations: The role of interaction and spatial abilities. Int. J. Digit. Earth 2017, 1–17. [Google Scholar] [CrossRef]
  83. Opach, T.; Popelka, S.; Doležalová, J.; Rod, J.K. Star and Polyline Glyphs in a Grid Plot and on a Map Display: Which Perform Better? Cartogr. Geogr. Inf. Sci. 2017. [Google Scholar] [CrossRef]
  84. Kukaňová, M. Porovnání dvou typů vizualizací z hlediska percepční a kognitivní zátěže a kognitivních schopností jedince. Comparison of the two Types of Visualization in Terms of Perceptual and Cognitive Load, and Personal Cognitive Abilities. Ph.D. Thesis, Masaryk University, Brno, Czech Republic, September 2017. [Google Scholar]
  85. Herman, L.; Řezník, T. 3D Web Visualization of Environmental Information–Integration of Heterogeneous Data Sources when Providing Navigation and Interaction. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Mallet, C., Paparoditis, N., Dowman, I., Elberink, S.O., Raimond, A.-M., Sithole, G., Rabatel, G., Rottensteiner, F., Briottet, X., Eds.; Copernicus GmbH: Gottingen, Germany, 2015; Volume XL-3/W3, pp. 479–485. [Google Scholar] [CrossRef]
  86. Herman, L.; Stachoň, Z. Comparison of User Performance with Interactive and Static 3D Visualization–Pilot Study. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Halounová, L., Li, S., Šafář, V., Tomková, M., Rapant, P., Brázdil, K., Shi, W., Anton, F., Liu, Y., Stein, A., et al., Eds.; Copernicus GmbH: Gottingen, Germany, 2016; Volume XLI-B2, pp. 655–661. [Google Scholar] [CrossRef]
  87. Herman, L.; Stachoň, Z.; Stuchlík, R.; Hladík, J.; Kubíček, P. Touch Interaction with 3D Geographical Visualization on Web: Selected Technological and User Issues. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Dimopoulou, E., van Oosterom, P., Eds.; Copernicus GmbH: Gottingen, Germany, 2016; Volume XLII-2/W2, pp. 33–40. [Google Scholar] [CrossRef]
Ijgi 06 00407 g001 550
Figure 1. Basic functional components of the Hypothesis platform (modified and extended from Štěrba et al. [42]): The first part, Database (see Section 4); the second part, Module Manager (see Section 2.4) and Export Module (see Section 2.4.4), Administration Account (see Section 2.4.1) with the Pack Menu available (see Section 2.4.3) and a Slide Editor (see Section 2.4.2); the third part, user interface of the web browser, allowing for data collection using a special SWT browser (see Section 2.4.3); and the fourth part, Selection Application (see Section 2.4.5).
Figure 1. Basic functional components of the Hypothesis platform (modified and extended from Štěrba et al. [42]): The first part, Database (see Section 4); the second part, Module Manager (see Section 2.4) and Export Module (see Section 2.4.4), Administration Account (see Section 2.4.1) with the Pack Menu available (see Section 2.4.3) and a Slide Editor (see Section 2.4.2); the third part, user interface of the web browser, allowing for data collection using a special SWT browser (see Section 2.4.3); and the fourth part, Selection Application (see Section 2.4.5).
Ijgi 06 00407 g001
Ijgi 06 00407 g002 550
Figure 2. The content of Slide, Task and Branch Tables is associated with a higher level in the hierarchy through the following intermediary tables: Task_Slide, Branch_Task and Pack_Branch.
Figure 2. The content of Slide, Task and Branch Tables is associated with a higher level in the hierarchy through the following intermediary tables: Task_Slide, Branch_Task and Pack_Branch.
Ijgi 06 00407 g002
Ijgi 06 00407 g003 550
Figure 3. Database tables with hierarchical relations between Slide Templates, Slide Contents and Task slides. The slide template table contains Slide Templates defining the character of Slides. Linking of Slide Templates to Slide Contents is ensured by GUIDs. Typically, several Slide Contents are linked to a single Slide Template.
Figure 3. Database tables with hierarchical relations between Slide Templates, Slide Contents and Task slides. The slide template table contains Slide Templates defining the character of Slides. Linking of Slide Templates to Slide Contents is ensured by GUIDs. Typically, several Slide Contents are linked to a single Slide Template.
Ijgi 06 00407 g003
Ijgi 06 00407 g004 550
Figure 4. (a) Structure of Pack No. 5 with only one branch (5). The branch contains two Sections (Tasks) (3 and 4) with several linearly arranged Slides; (b) Pack No. 6 with multiple branches.
Figure 4. (a) Structure of Pack No. 5 with only one branch (5). The branch contains two Sections (Tasks) (3 and 4) with several linearly arranged Slides; (b) Pack No. 6 with multiple branches.
Ijgi 06 00407 g004
Ijgi 06 00407 g005 550
Figure 5. Multi-task mode scheme: while completing a primary task displayed on the central, mouse-operated screen (Master Pack), the participant uses keyboard to react to target stimuli presented on peripheral screens (Slave Packs).
Figure 5. Multi-task mode scheme: while completing a primary task displayed on the central, mouse-operated screen (Master Pack), the participant uses keyboard to react to target stimuli presented on peripheral screens (Slave Packs).
Ijgi 06 00407 g005
Ijgi 06 00407 g006 550
Figure 6. Use of the Multi-player Mode in a computerized adaptation of Asch conformity experiments. First, all three participants were simultaneously shown an identical object; then, they were asked to estimate its size by drawing a corresponding line. The line was then displayed to all the participants.
Figure 6. Use of the Multi-player Mode in a computerized adaptation of Asch conformity experiments. First, all three participants were simultaneously shown an identical object; then, they were asked to estimate its size by drawing a corresponding line. The line was then displayed to all the participants.
Ijgi 06 00407 g006
Ijgi 06 00407 g007 550
Figure 7. Sample Administrator Account. Editing of a new user account.
Figure 7. Sample Administrator Account. Editing of a new user account.
Ijgi 06 00407 g007
Ijgi 06 00407 g008 550
Figure 8. Opening of a Pack in a controlled (featured) or standard (legacy) mode.
Figure 8. Opening of a Pack in a controlled (featured) or standard (legacy) mode.
Ijgi 06 00407 g008
Ijgi 06 00407 g009 550
Figure 9. Export Module with finished tests (marked).
Figure 9. Export Module with finished tests (marked).
Ijgi 06 00407 g009
Ijgi 06 00407 g010 550
Figure 10. Scheme of the technical structure (adapted from Šašinka et al. [20]).
Figure 10. Scheme of the technical structure (adapted from Šašinka et al. [20]).
Ijgi 06 00407 g010
Ijgi 06 00407 g011 550
Figure 11. Scheme of the whole experimental design (adapted from Kukaňová [84]).
Figure 11. Scheme of the whole experimental design (adapted from Kukaňová [84]).
Ijgi 06 00407 g011
Ijgi 06 00407 g012 550
Figure 12. Topographic map (a) and contextual transport map (b).
Figure 12. Topographic map (a) and contextual transport map (b).
Ijgi 06 00407 g012
Ijgi 06 00407 g013 550
Figure 13. Yellow lines (thickness) express frequency of the routes chosen by the experimental group (green digits) and control group (blue digits) together. Data from both groups are presented at once in the transport map.
Figure 13. Yellow lines (thickness) express frequency of the routes chosen by the experimental group (green digits) and control group (blue digits) together. Data from both groups are presented at once in the transport map.
Ijgi 06 00407 g013

Share and Cite

MDPI and ACS Style

Šašinka, Č.; Morong, K.; Stachoň, Z. The Hypothesis Platform: An Online Tool for Experimental Research into Work with Maps and Behavior in Electronic Environments. ISPRS Int. J. Geo-Inf. 2017, 6, 407. https://doi.org/10.3390/ijgi6120407

AMA Style

Šašinka Č, Morong K, Stachoň Z. The Hypothesis Platform: An Online Tool for Experimental Research into Work with Maps and Behavior in Electronic Environments. ISPRS International Journal of Geo-Information. 2017; 6(12):407. https://doi.org/10.3390/ijgi6120407

Chicago/Turabian Style

Šašinka, Čeněk, Kamil Morong, and Zdeněk Stachoň. 2017. "The Hypothesis Platform: An Online Tool for Experimental Research into Work with Maps and Behavior in Electronic Environments" ISPRS International Journal of Geo-Information 6, no. 12: 407. https://doi.org/10.3390/ijgi6120407

APA Style

Šašinka, Č., Morong, K., & Stachoň, Z. (2017). The Hypothesis Platform: An Online Tool for Experimental Research into Work with Maps and Behavior in Electronic Environments. ISPRS International Journal of Geo-Information, 6(12), 407. https://doi.org/10.3390/ijgi6120407

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop