OBSTACLE CHARACTERIZATION IN A GEOCROWDSOURCED ACCESSIBILITY SYSTEM

: Transitory obstacles – random, short-lived and unpredictable objects – are difficult to capture in any traditional mapping system, yet they have significant negative impacts on the accessibility of mobility-and visually-impaired individuals. These transitory obstacles include sidewalk obstructions, construction detours, and poor surface conditions. To identify these obstacles and assist the navigation of mobility-and visually-impaired individuals, crowdsourced mapping applications have been developed to harvest and analyze the volunteered obstacles reports from local students, faculty, staff, and residents. In this paper, we introduce a training program designed and implemented for recruiting and motivating contributors to participate in our geocrowdsourced accessibility system, and explore the quality of geocrowdsourced data with a comparative analysis methodology.


INTRODUCTION
Mapping dynamic geographic phenomena is often difficult, due to the requirements for frequent updates and changes that occur over time.In urban areas, pedestrian corridors and transportation infrastructure are the most in-demand, critical features to map, yet they are frequently impacted by sidewalk obstructions, construction detours, and changing surface conditions (Figure 1).For individuals with vision or mobility impairments, changes to the pedestrian corridors (even temporary ones) are very difficult, due to the necessity for rerouting, inconvenience, and the increased risks associated with safety hazards.Mapping these areas with high-frequency coverage is essential.Where remote sensing and automated feature extraction is not a realistic approach due to cost or required frequency of updates, a different approach is needed.Rein (2009) provides evidence of the great cost and difficulty associated with improving and maintaining a functional, accessibly multimodal transportation network.Beale et al. (2006), Laakso et al. (2011Laakso et al. ( , 2013)), Karimi et al. (2013), and Kasemsuppakorn et al. (2009) provide useful approaches for pedestrian network modeling and applications for routing disabled individuals.Nuernberger (2008) and Barbeau et al. (2010) show how communication devices can be used to enhance navigation and improve communication with disabled individuals.The primary missing element in these approaches is the ability to capture and map transient events, which is difficult due to their dynamic nature.Goodchild (2007Goodchild ( , 2009) )  The work presented here uses data contributed by the public (VGI) but also enhances the active use of this data through active map-based routing and other services (VGS).A critical aspect of the provision of services is an active quality assessment system, which depends on an understanding of the obstacle characterization abilities of the data contributors.The following sections of this paper present an overview of the GMU Geocrowdsourcing Testbed and its moderation and quality assessment program, the training and obstacle characterization studies, a short discussion of user motivations, a summary of positional accuracy characterization, and finally, conclusions and future work.

THE GMU GEOCROWDSOURCING TESTBED
The GMU Geocrowdsourcing Testbed (GMU-GcT), presented in Rice et al. (2014), was developed as an experimental approach for mapping transient navigation obstacles in the region surrounding the George Mason University Campus in Fairfax, Virginia.The GMU-GcT uses crowdsourced data contributions from members of the public, who identify, document, position, and describe obstacles through a mapbased crowdsourcing approach.This system is built on the work of Paez (2014) and other best practices for map-based geocrowdsourcing.The system uses a menu-driven, semistructured reporting process where end-users provide location information, temporal tags, images, and attribute data for obstacles in the Fairfax, Virginia region (Figure 1).The GMU-GcT has both desktop and mobile contribution tools to facilitate obstacle reporting by the public and provides services such as obstacle-avoidance routing (Qin et al., 2015). 3

Moderation and Quality Assessment in the GMU Geocrowdsourcing Testbed
Many authors have explored data quality issues in geocrowdsourcing, including Haklay (2010) and Girres et al. (2010) who explore the accuracy of crowdsourced features in OpenStreetMap through comparisons with authoritative Ordnance Survey and French IGN data.The assessments in Girres et al. (2010) are noteworthy for the thoroughness of the evaluation in the context of well-known map accuracy and GIS data accuracy characterization methods, such as those published by Guptill et al. (1995), Hunter et al. (1992), and Veregin (1999).Goodchild and Li (2012) suggest different methods for quality assessing geocrowdsourced data.Because of the unofficial nature of the data, the GMU-GcT uses Goodchild and Li's social approach, where a team of trained moderators checks data contributions for errors.This quality assessment system is discussed in Rice et al. (2014) and Paez (2014).A key to the data quality approach in the GMU-GcT is the ability of end-users to accurately identify, document, and characterize obstacles.While some elements of data contribution, such as map-based obstacle positioning, are objective and easy to analyze for errors, other elements are more difficult.Obstacle characterization in the GMU-GcT requires a shared understanding of obstacle categories between the system administrators and system end-users.This categorical characterization process, as noted by Camponovo et al. (2014) in the context of the Haiti Earthquake response, is problematic.They report that over fifty percent of the messages to the Ushahidi web platform were mischaracterized with regard to emergency need.Foody et al. (2013) develop approaches for assessing the relative accuracies of end-user categorization of land cover.Galloway et al. (2013) tested the ability of young contributors to identify tree species, finding the contributors over-reported rarer classes.Quality assessment for crowdsourced geospatial data is a challenge, as noted by many of the fore-mentioned authors.The approach for quality assessment used in the GMU-GcT is built on the best practices for these approaches, with Goodchild and Li's moderator-based social quality assessment as the model.

Training and Obstacle Categorization in the GMU Geocrowdsourcing Testbed
To improve the quality of information contributed to the GMU-GcT, Paez (2014) conducted a thorough review of training strategies in map-based social applications and geocrowdsourcing and found the most effective methods of training to be those that were embedded within the data contribution tools, such as those embedded within Google Map Maker and OpenStreetMap.Based on Paez's work, training videos are embedded within the GMU-GcT, through the "How it Works" link. 4 While effectiveness or success of the training methods can be difficult to assess, Paez (2014) provides a summary and assessment of best practices.
For the GMU-GcT, the primary means of characterizing transient navigation obstacles is through the placement of a map-based location icon, the categorization of the obstacle in question, and categorical assessments of the obstacles expected duration and urgency.While the positioning of obstacles is of high interest (Rice et al., 2015), a critical aspect is the contributor's obstacle categorization.Potential contributors to the GMU-GcT (which includes a wide range of students, faculty, staff, and local community members) were asked to participate in a training exercise to learn obstacle categorization, and were shown a series of pictures with authoritative obstacle characterizations shown (Figure 2).The general obstacle types shown in Figure 2 and used in the GMU-GcT (sidewalk obstruction, construction detour, entrance/exit problem, poor surface condition, crowd/event) were derived from an end user study conducted in 2013 and reported in Rice et al. (2013a).A subsequent training step asked potential contributors to identify the obstacle type from a simple unlabeled picture (Figure 3) using the obstacle types presented in the previous step.While some obstacle pictures were relatively easy to characterize (Figure 3, Figure 4) others were much more complex (Figure 5, Figure 6).Figures 9 and 10 show relatively low levels of disagreement for the category of the object shown in Figure 9, which may be attributable to the less complex scene.A majority of participants categorized the obstacle in this image as a poor surface condition.Figure 11 shows the frequency of all obstacle category tags in the GMU-GcT as of January 2014.

User motivation in the GMU Geocrowdsourcing Testbed
The success of many geocrowdsourcing projects depends on the motivations and willingness of participants.For OpenStreetMap, an initial motivation for participation is described by Coast (2006) as resentment over the data pricing and data licensing practices of the Ordnance Survey.Furthermore, Coleman et al. (2009) note that differences in user needs and motivations are based on the different contexts in which they contribute.A study by Pease (2014) underscores the potential problems associated with user motivation in geocrowdsourcing.While typical positional accuracy levels for the GMU-GcT are in the same range as feature position accuracy positions noted by Haklay (2010) and Girres et al. (2010) for OSM data (Rice et al., 2014), Pease noted lower levels positional accuracy, and a higher level of disagreement for obstacle categorizations.In the study by Pease (2014), students were asked to characterize the position and attributes of obstacles along a path.The obstacles had previously been thoroughly characterized by trained moderators and project staff.Pease noted an average positional error of 13.07 meters for obstacle reports, and greater level of disagreement for obstacle categorization than for similar reports collected by the same GMU-GcT from the same area.The differences, according to project staff (Paez, 2014;Rice et al., 2014) were due to the motivation of the study participants.While in general, participation in the GMU-GcT is based on altruism and a desire to correct accessibility problems, the study participants in Pease (2014) were recruited from a general subject pool where extra credit was offered, which was suggested as a reason for their performance.

POSITION ACCURACY
The GMU-GcT uses the positioning of reported obstacles in order to facilitate obstacle avoidance routing on a pedestrian network, which is motivated by the accessibility mapping and accessibility wayfinding work of Golledge (1999), Golledge et al. (2000Golledge et al. ( , 2005Golledge et al. ( , 2006)) 12), obstacle positioning from a movable location icon (yellow marker in Figure 12) and positioning determined through a social moderation of the report (green marker, Figure 12).This work continues, with recent efforts to use electronic compass data embedded in image headers to validate the positions of obstacles (Rice et al., 2015).
Figure 12.Positional characteristics of an obstacle report

CONCLUSION AND FUTURE WORK
The GMU-GcT has been built to gather information about transient navigation obstacles in the local environment.Gathering these obstacles using a geocrowdsourcing system provides many benefits, including the ability for blind, visuallyimpaired, and mobility-impaired individuals to be informed in advance about unexpected changes to a pedestrian pathway.It will also allow these individuals to re-route, and avoid risks and significant delays.The data quality assessment is done through a social moderation process and based on best practices.The ability of data contributors to characterize and categorize obstacles is important to the system, and has been studied and improved through an iterative process and feedback from endusers and contributors (Paez, 2014;Rice et al., 2014).The GMU-GcT allows for obstacle avoidance routing, and to do so requires obstacle data whose positional characteristics and attributes are well known.Project researchers will continue to improve the system and maintain high quality geographic information.Critical future work will refine the routing elements and active-use elements of the system, including mobile data validation tools and mobile moderation and quality assessment tools.A recent publication by Karimi et al. (2014) offers insights and strategies in a similar project, which will provide opportunities to adopt best practices and workflows from a different geographic setting.

Figure 2 .
Figure 2. Training graphic with picture and obstacle type, obstacle duration, and obstacle priority characterization

FigureFigure 8 .
Figure 7. GMU geocrowdsourcing training picture showing sidewalk obstruction , Church et al. (2003),Jacobson (1998),Pingel (2010) andRice et al. (2005).Implemented in the system are four methods of evaluating positional accuracy: (1) human-georeferenced geographic coordinates via computer, (2) mobile GPS coordinates from user's current position, (3) embedded geo-tags in an image of reported obstacle, and (4) convex hulls created from geoparsed text descriptions of the obstacle's location, based on a comprehensive gazetteer.This work, characterized inRice et al. (2015), involves the development of metric georeferenced footprints from geoparsed location descriptions (red outline in Figure12, based onRice et