GeoData2014 Post-Mortem

Blog post
added by
Wiki Admin

The workshop theme and community output notes may be of high interest. The focus was more on how Federal Geodata “Operations” / Assets can improve and help Geoscientists through improved interagency coordination. 
There are excellent breakout notes on roadblocks, geoscience perspective, concrete steps, etc. across the following topics on the following URL (Google Docs under “notes” links): http://tw.rpi.edu/web/Workshop/Community/GeoData2014/Agenda
Topics:
Day 1 Breakouts (Culture/Management)
Governmental open data 
Interagency coordination of geodata – progress and challenges
Feedback from the academic and commercial sectors
Collaborating environment and culture building
Day 2 Breakouts (Tech)
Data lifecycle
data citation and data integration frameworks – technical progress
Experience and best practices on data interoperability 
Connections among distributed data repositories – looking forward
Raw Notes:
The workshop has some fruits coming out of it. 
  • About 50 people. NOAA and USGS on Fed side primarily. 
  • Pushing forward on agenda to see if we have progressed on ideation pragmatism since Geodata2011.
  • Focus is on Cultural and Financial issues limiting inter-agency connection. 
  • Term agile government came up often… with some laughs, but some defenders (Relates to our smartleangovernment.com efforts with ACT-IAC)
  • Scientists hear Architecture as Big IT contracts and IT infrastructure, not process improvement, data integration, goal/mission alignment, etc., so there is clear vernacular issues.
  • FGDC and tons of other standards/organizing bodies seen as competing and confusing
  • data.gov and open data policy hot topic (Seen as good steps, low quality data) – “geoplatform” mentioned exactly “zero” times (doh!)
  • geo data lifecycle primarily on the latter end of cycle (citations, discovery, publication for reusability, credit) but not much on the project coordination, data acq coordination, no marketplace chatter, little on coordinating sensor investment
  • General questions on how scientists were interested on how intel groups can be reached 
  • Big push on ESIP
  • Concrete steps suggested were best practices to agencies and professors
  • Data Management is not taught, so what do we expect? You get what you pay for.
  • Finally, big push on how to tie grassroots efforts and top-down efforts together – grassroots agreed we need to showcase more, earlier, and get into the communities top-down folks are looking at.
  • Not high Federal representation there, and agreed with limited Government travel budgets, we need to bring these concepts to them and meet on their meetings, agendas, conferences, circuits, and push these concepts and needs.

Again, lots of great notes from breakouts on roadblocks, geoscience perspective, concrete steps, etc. across the following topics on the following URL (Google Docs under “notes” links): 

http://tw.rpi.edu/web/Workshop/Community/GeoData2014/Agenda

Questions we posed in general sessions:
  • Of Performance, Portfolio, Architecture, Evangelism, Policy, or other what is more important to the GeoScientist that needs to be addressed in order to improve inter-agency coordination
  • You noted you want to truly disrupt or re-invent the motivation and other aspects in the culture, what was discussed related to doing such – inter-agency wiki commons a la intelipedia? Gamification and how incents resource Management MMORPGs – i.e. transparent and fun way to incentivize data maturity? Crowdsourcing a la mechanical turk to help in cross-agency knowledge-sharing? Hackathon/TC Disrupt Competitions to help in showcase? Combindation – i.e. Gamify metadata lifecycle with crowd model?
  • After registering data, metadata, good citations, and doing all the data lifecycle management, and if we are to “assume internet”, who is responsible for the SEO Rank to help people find scientific data in the internet – who assures/enhances schema.org registrations? who aligns signals to help with keywords, and thousands of other potential signals? especially in response to events needing geoscience data? who helps push data.gov and domain catalogs to be harvested by others?

Exploring Public Domain Maps and Imagery – Historic Denver West

Blog post
edited by
Matt Tricomi

Doing some light examination of Denver West, the following is just two aspects of free public domain data. There is so much more beyond imagery and maps, and even more maps and more imagery (i.e. MODIS, LANDSAT, other NASA products).

Historic Maps

USGS Topo Map 1-meter Quads are clockwise:

(Download this KML file to see all options or click the years above the thumbnails below to grab that historic map)

Golden – Download – 1939194219441957

6 Diff dates for 15 Quads 1939-1965

Arvada – Download – 19411944195019571965

5 Diff dates for 12 Quads 1941-1965 

Morrison – Download – 19381942194719571965

5 Diff dates for 11 Quads 1938-1965 

Fort Logan – Download – 1941194819571965

4 Diff dates for 8 Quads 1939-1965 

Historic Imagery

Historic Imagery typeYearMetadataThumbnail (Actual Files range from 20-200 MB)
Digital Orthophoto Quadrangles (DOQs)1994
  • Entity ID: DI00000000912683
  • Acquisition Date: 23-SEP-94
  • Map Name: FORT LOGAN
  • State: CO

 

 

 
National Aerial Photography Program (NAPP)1988
  • Entity ID: NP0NAPP001033224
  • Coordinates: 39.8125 , -105.03125
  • Acquisition Date: 10-JUL-88
 
National High Altitude Photography (NHAP)1983
  • Coordinates: 39.671135 , -105.043794
  • Acquisition Date: 25-JUN-80
  • Scale: 120400

Download ~30MB file

 

Space Acquired Photography   

Single Frame Records

Black-and-white, natural color, and color

infrared aerial photographs

400 or 1,000 dpi.

1978
  • Entity ID: AR1VEQP00010142
  • Coordinates: 39.688989 , -105.053705
  • Acquisition Date: 01-SEP-78
  • Scale: 78000
Aerial Photo Mosaics (Used when creating early/mid-USGS Topo Maps)1953
  • Entity ID: ARDDA001260930776
  • Coordinates: 39.5 , -105.5
  • Acquisition Date: 25-SEP-53
  • Scale: 63299

Download the 30MB file

 

    
High Resolution OrthoImagery (Corrected – Generally .3 meter, color) 

2002 – HistoricOrthoCoverageAreas.kml

 

 
Declass 1 (1996) Stereo Images1965
  • Entity ID: DS1027-1015DA011
  • Coordinates: 39.69 , -104.516
  • Camera Resolution: Stereo Medium
  • Acquisition Date: 10-DEC-65
Declass 2 (2002) Stereo Images1966
  • Entity ID:DZB00403500080H001015
  • Coordinates: 42.49 , -103.45
  • Acquisition Date: 10-DEC-66
  • Camera Resolution: 2 to 4 feet

Macro versus Micro Geospatial Data Value

Blog post
edited by
Matt Tricomi

Kudos to the Canadian Government – NRCAN – for trying to get an clearer understanding of the economic significance and industry status of  geospatial data products, technologies and professional services industry.  http://www.nrcan.gc.ca/earth-sciences/geomatics/canadas-spatial-data-infrastructure/cgdi-initiatives/canadian-geomatics-0.  The Boston Consulting Group (BCG) produced a somewhat similar economic impact assessment for the United States in April 2013. http://geospatial.blogs.com/geospatial/2013/04/contribution-of-geospatial-services-to-the-united-states-economy.html

We geo-bigots working to support the public sector intuitively and intellectually realize geospatial’s potential value and are continuously frustrated by our lack of collective ability and capacity to make its implementation more immediate, simpler and more powerful.  While these macro- economic pictures are interesting and useful they do not seem to influence the micro government decision making capacities.  The BCG report “cautions that to continue this growth will require sustained public- and private-sector cooperation and partnership, open policies governing collection and dissemination of location-based data and increased technical education and training at all level”.

The intent of the US Federal Geographic Data Committee  (FGDC) “GeoPlatform” and Canada’s FGP, if supported by the proper policies and data management practices could simplify the data quality and acquisition challenges resident in our hyper-federated geospatial data environment.  Ironically, in the emerging and soon to be overwhelming information and knowledge based economy we are still struggling to manage data content.  Geospatial will not, break into the program and mission operations until the business leadership fundamentally adopts information centered performance objectives as a part of their organizational culture. 

Geospatial has always been an obtuse concept to classify, evaluate or pigeonhole into a nice neat framework let alone determine its national economic value.  For similar reasons, within the US Federal government, “geospatial” has struggled to find an organizational position that would enable its potential value to be maximized.  This is partly due to data structure complexity resulting from its form, resolution, temporal range, scale, geometry or accuracy qualities have created an artificial “boundary” and organizations are having are hard time navigating out of.  At least until the “director” sees the “map” or the “picture’ and then it is the silver bullet.

Here at Xentity, we want to start to frame the discussion about how to exploit Geospatial Value at the micro or organizational level and begin to guide our customers to sustained geospatially driven business improvements.  Our initial cut at how to break the field down is found in the diagram.  How can this be improved upon?

In 2014 Every Business will be Disrupted by Open Technology

Blog post
added by
Wiki Admin

The article “In 2014 Every Business will be Disrupted by Open Technology” raised some very key points on some key disruptive patterns. A key theme we picked up on was the movement from bleeding and leading edge to more popular adoption of the open data and Open platforms. 

As the article notes:

Yet the true impact begins not with invention, but adoption.  That’s when the second and third-order effects kick in. After all, the automobile was important not because it ended travel by horse, but because it created suburbs, gas stations and shopping malls

A few tangible themes we picked up on are:

  1. Stand-alone brands are shifting to open platforms for co-creation
  2. Open platforms built on a brand`s intellectual property enhance the value, not threaten it
  3. Open platforms provide enterprise-level capabilities to the smallest players, leveling the playing field, accelerating innovation, and amplifying competition
  4. Open platforms for co-creation shifts the focus away from driving out inefficiencies, and toward the power of networking and collaboration to create value.

Where its happening already: Federal, State, City, Commercial

In FedFocus 2014, they also emphasized that with the budget appropriations for 2014 and 2015, two big disruptive areas will continue to be in Open Data and Big Data. Especially, with the May 2013 release of the WhiteHouse Open Data memorandum for going into effect in November 2013, it will impact Open Data by:

Making Open and Machine Readable the New Default for Government Information, this Memorandum establishes a framework to help institutionalize the principles of effective information management at each stage of the information`s life cycle to promote interoperability and openness. Whether or not particular information can be made public, agencies can apply this framework to all information resources to promote efficiency and produce value.

We are seeing states get into the mix as well with Open Data movements like http://gocode.colorado.gov/

Go Code Colorado was created to help Colorado companies grow, by giving them better and more usable access to public data. Teams will compete to build business apps, creating tools that Colorado businesses actually need, making our economy stronger.

Also, at the city level with the City of Raleigh, North Carolina which is well recognized for its award-winning Open Data Portal.

 

We had previously tweeted on how IBM opened up their Watson cognitive computing API for developers…. publicly. This is a big deal. They know with open data platforms as an ecosystem, they not only get more use, which means more comfort, which means more apps, but every transaction that happens on it, that is legally allowed, they to improve their interpretative signals that make Watson so wicked smart. This article points this key example out as well. 

 

And back to National Data Assets moving ahead to make their data more distributable over the cloud, moving data closer to cloud applications, offering data via web services where they are too large or updated too often to sync, download, or sneakernet.

Xentity and its partners have been at the forefront of all these movements.

We have enjoyed being on the leading edge since the early leading edge phases of this movement. Our architectures are less on commodity IT, which not to undersell the importance of affordable, fast, robust, scalable, enabling IT services and data center models. Our architectures have been more focused on putting the I back in IT.

We have been moving National Geospatial Data Assets into these delivery models as data products and services (Xentity is awarded USGS IDIQ for Enterprise and Solution Architecture), supporting the architecture of data.gov (Xentity chosen to help re-arch data.gov), and recently supporting the data wrangling on Colorado`s State OpenData efforts. We are examining Can a predictable supply chain for geospatial data be done and actively participating in NSF EarthCube which looks to “Imagine a world…where you can easily plot data from any source and visualize it any way you want.” We have presented concepts

Our architecture methods (Developing a Transformation Approach) are slanted to examine mission oriented performance gains, process efficiencies, data lifecycle management orientation, integrating service models, and balancing the technology footprint while innovating. For instance, we are heavily involved in the new ACT-IAC Smart Lean Government efforts to look at aligning services across government and organizational boundaries around community life events much like other nations are beginning to move to.

Xentity is very excited about the open data movements and supported platforms and the traction it is getting in industry. This may move us forward from information services into the popular space to and for knowledge services (Why we focus on spatial data science)


To do BigData, address Data Quality – People and Processes – Tech Access to information

Blog post
added by
Wiki Admin

As a follow on to the “cliffhanger” on BigData is a big deal because it can help answer questions fast, there are three top limitations right now: Data Quality, People and Process, Tech Access to Information. 

Lets jump right in.

Number One and by far the biggest – Data Quality

Climate Change isn’t a myth, but it is the first science to ever be presented on a data premise. And in doing so, they prematurely presented models that didn’t take into account the driving variables. Their models have changed over and over again. Their resolution of source data has increased. Their simulations on top of simulations have proven countless theories of various models that can only be demonstrated simply by Hollywood blockblusters. Point being, we are dealing with inferior data for a world scale problem, and we jump into the political, emotional driven world with a data report? We will be the frog in slowly warming water, and we will hit that boiling point late. All because we started with a data justification approach using low quality data. Are they right the world is warming? Yes. Do they have enough data to proven the right mitigation, mediation, or policy adjustments? No, and not until either we increase the data quality or take a non-data tact.

People and processes is a generation away.

Our processes in IT have been driven by Defense and GSA business models from the fifties. Put anyone managing 0s and 1s technology in the back. They are nerds, look goofy, can’t talk, don’t understand what we actually do here and by the way, they smell funny. That has been the approach to IT since the 50s – nothing has changed with the exception that their are a few bakers dozen of the hoodie wearing, mountain dew drinking, late night owls who happen to be loaded now, and their is a pseudo culture of geek chic. We have not matured our people talent investment to balance maturity of service, data, governance, design, and product lifecycle to embrace that engine culture as core to the business. This means, more effective information sharing processes to get the right information to the right people. This also means, investing in the right skills – not just feeding doritos and free soda to hackers – to manage the information sharing and data lifecycle. I am not as worried about this one. As the baby boomer generation retires, it will leave a massive vacuum as Generation X is too small and we’ll have to groom Generation Y fast. That said, we will mess up a lot missing a lot of brain drain, but market will demand relevancy which will, albeit slowly, create this workforce model in 10-15 years.

Access to Environments 

If you asked this pre-hosting environments or pre-cloud, this would have been limited to massive corporations, defense, intel, and some of the academia co-investing with those groups. If you can manage the strain of shifting to a big data infrastructure, this barrier should be the least of your problems. If you can allow your staff to get the data they need at the speed they need so they can process in parallelization without long wait times, you are looking good. Get a credit card, or if Government, buy off a Cloud GWAC, and get your governance and policies moving, as they are likely behind and not ready. Likely they will prolong the silo’d information phenomenon. Focus on the I in IT, and let the CTO respond to the technology stack. 

Focus on data quality, have a workforce investment plan, and continue working your information access policies

The tipping point that move you into Big Data is where these combined require you to deal with the complicated enormity at speeds answering questions not just for MIS and reports, but to help answer questions. If you can focus on those things in that order (likely solving in reverse), you will be able to implement parallelization of data discovery.

This will shorten the distance from A to B and create new economies, new networks, and enable your customer or user base to do things they could not before. It is the train, plane, and automobile factor all over again.

And to throw the shameless plug in, this is what we do. This is Why we focus on spatial data science and Why is change so fundamental.