GeoSandbox 2.0

This Week’s Project

I decided it was time to take the GeoSandbox to the next level. As usual, I had multiple goals in mind for this project. Primarily, I wanted to learn how to use and write JavaScript. It’s been a long time since I’ve tried to write any kind of meaningful code, but I figured if I wanted to keep up with the big kids, it was time to take the plunge. I also wanted to get my map viewer into a state where I could make it useful enough to display my GIS projects, and help promote my business.

My Experience with Writing Code

My first exposure to computer programming was back in 1980 when I took a computer language class as an undergrad. I remember trying to write some code for a fictional car dealership inventory system, and saving it on a 8” floppy disc. I believe it was in Basic, but I really can’t remember. I never did get it to work. Since then I’ve played around with various versions of Visual Basic, but never got very far past the “Hello World” types of programs. I took a GIS class as a grad student back in 2001 where I used MapObjects to make a functional GIS program. That was fun, and I did get that to work, but I haven’t written any real code since then. Yes, I do read, use, and modify scripts in ArcGIS. That keeps me in tune with the basics of programming, but I don’t think it’s the same as writing real code.

The Goal

I wanted to make the following three enhancements to the GeoSandbox:

  • Change the buttons in the toolbar
  • Make the map window dynamically change size with changes to the browser window size
  • Have the map window change its contents through links in order to showcase different projects

The first two items were fairly easy. I was able to read through the GeoExplorer.js script, and just delete the buttons I didn’t want showing on the toolbar. I also made some changes to the About box, and figured out how to generate a proper Google API key that allows the embedded 3D viewer to show. Dynamically resizing the map window was just a matter of doing a search, copying some code, and pasting it in the right place on the web page. Yes, I had to modify the <iframe>  ID in the code, but that was simple enough.
I had it in my head that I wanted to make four maps available on the page, and to allow users to switch between these maps using a set of links along the top of the map. Seemed simple enough.

No, it wasn’t.

Setting up the webpage and the GeoServer maps was easy. Here’s what the page looks like:

GeoSandbox2-0

And you can access the page using this link: www.donmeltz.com/maps

Changing iFrames

What I found out along the way was, banging your head against an <iframe> can really hurt.

I started out by putting a single <iframe> on the page. Since the src attribute of said <iframe> specifies the URL of the document to show in the <iframe>, I figured I’d just change the src attribute to point to the various GeoServer maps I wanted to show. That didn’t seem to work for me. What I found was, there are many different ways to change the <iframe> src. I must have tried them all, and I got most of them to work. The problem was, they didn’t work in every browser I tried. Here’s a list of the browsers I used in my tests:

  • Firefox 3.1.15
  • Internet Explorer 9 beta
  • Chrome 9.0.597.107
  • Safari 5.0.3
  • Opera 10.10

Learning About iFrames

First, I’d like to mention the Dynamic Web Coding website has the most comprehensive and understandable information on scripting <iframes> that I’ve found: http://www.dyn-web.com/tutorials/iframes/
What I haven’t figured out yet is – Why do all of the examples used on that website (and many others I visited) work for them, but not for me? It just boggles my mind. In a nutshell, here are the various things I tried.

I started out by setting up some bare-bones test sites where I stripped out all of the css styling, and deleted all but two links in order to make it easy to modify and test the code, and see what I was doing. I have links to all of these test pages below each of the following paragraphs in case you want to check them out. You can right-click on each page and view the source if you want. Click on the upper part of the page, NOT on the <iframe> map.
(Please note, I have greatly simplified the syntax in these examples so they fit on one line. If you want to see the actual code, please visit the Dynamic Web Coding site, or right click on my test maps and view the source.)

I began by trying to set the <iframe> target directly in the link like this:

<a href=http://www.Maps.html#maps/2 target="mapiframe">link</a>

This worked in IE and Firefox, but only if I hit the browser refresh button after I clicked the link. Yes, I also went through many iterations of adding a variety of refresh functions to the code, but that only worked in one or two instances. And in some cases it made things even worse. Here’s a link to my test page for this trial: http://www.donmeltz.com/mapstest1.html

Next up was an attempt to set the iframe source (scr attribute) using some JavaScript code as in:

document.getElementById('MapFrame').src="http://Maps.html#maps/2"

I tried using both a function and embedding the code in the link using the onClick event. Results were similar to the previous attempt, and again, inserting a refresh function did not help. There are other variations of this, such as using “window.getElementById, etc. which I also tried, but to no avail. Here’s the link to my test page for this trial:
http://www.donmeltz.com/mapstest2.html

I then moved on to trying to assign the contents of the iframe using code like this:

contentWindow.location.assign("http://Maps.html#/2")

which worked in IE (again, with a browser refresh) but not in anything else. The trial page: http://www.donmeltz.com/mapstest3.html

What came closest to getting the task done was working with the <div> that holds the <iframe> instead of manipulating the <iframe> directly, as in:

<div id="MapContainer">
MapContainer.innerHTML = "<iframe id="MapFrame" src="http://Maps.html#maps/2">

This one worked for me in every browser except Firefox: http://www.donmeltz.com/mapstest4.html

In the end, what I wound up doing was placing four different iframes on the page. I then set the style.display property of the iframe to “none” (hide) or “block” (show) in order to hide or show each one based on which link is clicked on the web page and looks something like this:

document.getElementById("MapFrame").style.display="block";

Yes, “block” means “show”. Intuitive, I know. Here’s a link to my test page that shows how this one works: http://www.donmeltz.com/mapstest_adinfinitum.html
This works in every browser I’ve tried, but it does slow down the load times significantly. All four maps load when the page loads. It does make switching between maps faster once everything is loaded, though.

I Got It

So, where did all of this hacking get me? I got what I wanted. A workable map viewer that shows a few samples of my work, and an education in JavaScript. Even though I spent many hours, days even, working with this, I still think it was worthwhile. I’ve learned a lot, even though it doesn’t seem like I got very far. I can read JavaScript code, and understand it a lot better now than I did when I started. I’d still like to figure out if I’m doing something wrong. It seems like these options should work, as it looks like they work on other peoples pages. Yes, I’m still banging my head against the <iframe> once in a while :-)

For now though, I think I’ll just kick back with a bottle of homebrew and a little Jimmy Cliff.

Posted in GIS | Tagged | 4 Comments

Mapping My Twitter Followers – No Code Needed

The Challenge

I’ve been pondering what new data sets to add to the GeoSandbox for a while now. It’s been a couple of weeks since the last addition, and I felt things were getting a little stale. Once again, a single tweet set my mind in motion. This time it was @briantimoney who said “2011 is the year tweet maps replace coffeeshop-finding as the go-to demo scenario in geo.” An RT by @billdollins cemented the statement in my head, and there was no going back after that. Obviously if the GeoSandbox is to remain the cutting-edge tool that it is, I’d have to move quickly.

The Process

All I wanted to do was to make a push-pin style map of the people who follow me on Twitter. Simple as that. I did not want to get into learning APIs and writing any kind of code, and I didn’t want to get into auto-updating, or anything like that. KISS. A little searching led to the following workflow:

MyTweeple – Export to CSV – Import to Fusion Table – Visualize on map –
Export to KML – Convert to shapefile – Put it into GeoServer.

More extensive searching might reveal a more efficient process, but this is what I came up with. From the time I first read the tweet to the point the data was in the GeoSandbox took about one hour.

My Tweeple to CSV

Once I signed into the MyTweeple site, I chose the Tools tab, and then the Export All (csv) option. This allowed me to save a csv spreadsheet containing up to 5,000 of my followers (I only have ~560). There are a lot of extraneous columns that can be done away with. All I really wanted was the name and address columns. Some sorting and editing of the address column can help out, too. Not everyone includes an address in their bio, and some have addresses that obviously won’t geocode properly. These were deleted.

MyTweeple

Fusion Table to KML

I’ve heard a lot of talk on the inter-tubes lately about Google Fusion Tables, so I’ve been looking for something to give this new tool a test drive. This was my chance, and it did not disappoint. All I had to do was go to the Fusion Table page, select New Table>Import Table, and browse to the MyTweeple.csv file I saved earlier.

FusionTable

I double checked a few addresses, and then clicked on the Visualize>Map link. The geocode routine ran for a few minutes, and produced a map showing all of the people who follow me on Twitter. It was like magic.

KMLmap

Clicking the Export to KML link allowed me to save the KML file to my computer, where I then converted it to a shapefile (I used the KML to Layer tool in ArcToolbox, but I’m sure there are many other ways to do this). From there it was just a matter of adding the shapefile to my GeoServer as I’ve outlined in previous posts.

The Results

GeoSandboxTwitterers

So there you have it. My somewhat convoluted way of showing my Twitter followers in the GeoSandbox.

Posted in GIS | Tagged | 6 Comments

GeoSandbox grows up to be a “Real boy”

A quick update on the status of my little experiment.

This morning I got it in my head that I wanted to access my GeoSandbox in ArcMap through WMS, and allow others access to it through the same mechanism. Through much trial and error, and some timely help from @mcbride_bryan, I was able to accomplish my goal. If you want to try loading any of the data on my GeoSandbox into your GIS client of choice, you should be able to do so through this URL:

http://donmeltz.dyndns.org:8080/geoserver/wms?service=wms&version=1.1.1&request=GetCapabilities

Here’s a screen-shot with all the current layers loaded:

GeoSandboxWMSinArcGIS

Thanks also to @tpstigers for verifying that he could access it in his GIS app of choice, QGIS running in Ubuntu:

GeoSandboxWMSinQGIS

I’m really surprised at how fast these layers load. They seem to load much faster through the WMS than they do in the webpage.

So, I am happy to report that my little GeoServer toy is well on its way to becoming a “Real Boy”

Posted in GIS | Tagged , | 2 Comments

Setting up my GeoSandbox

It’s been 3 weeks since I set up my little GeoServer experiment and started populating it with various datasets. I thought I’d outline the things I’ve been working on, and what I plan on looking at next. I couldn’t have come as far as I have without the generous help of some of my Twitter friends. I’ve noted below each section those who’ve helped me out. I apologize if I’ve overlooked anyone.

Workspace > Store > Layer

My initial step in getting a useful map up and running was to load some vector data. The first thing I’m sure every GeoServer noob learns is, importing data into GeoServer is a three-step process. The first thing I did was set up a Workspace, which is essentially a folder in my “C:\Users\<username>\.opengeo\data_dir\data\<foldername>” directory. Then I told GeoServer where to look for this folder. Second step was to set up a Store. A Store can hold a single layer, or multiple layers depending on how you import the data. Why a Workspace AND a Store? I have no idea, but that’s what you have to do, so that’s what I did. Third step was to add a layer or layers to the Store. After that, you can add those layers to a GeoExplorer map, and publish it or embed it in a webpage.

EPSG

Somewhere along the way, GeoServer will ask for a spatial reference system (SRS) for the data you’re adding in the form “EPSG:XXXX”. I figured out very quickly that looking up the EPSG for a shapefile using the built-in GeoServer tool is a PITA. A couple of tools were pointed out to me:
Prj2EPSG (http”//prj2epsg.org/search) and Spatial Reference (http://spatialreference.org)
Although the data I use comes to me in many different projections, I use a limited number of spatial references in my output. I now keep a Post-It note on my desk with the 5 EPSG codes I use most often.
(Thank you @mcbride_bryan, @spara)

Styling

Symbolizing vector layers in GeoServer is a little cumbersome, but I was able to figure it out. and it’s not too much of a chore for simple layers. Options are limited, and I’ve been pointed to a few other tools when it comes to styling layers. uDig, Atlas Styler, and Arc2Earth all look promising and I’ll be checking them out in the near future.
(Thank you @mcbride_bryan, @spara)

Rasters

One of my more frustrating episodes was an attempt to get raster data to display in my GeoExplorer map. The GeoServer GUI and documentation led me to believe I could just load an ArcGrid (binary) dataset into GeoServer. I wasn’t able to get that to work, and have been told by some that it’s not supported. Next, I tried a jpeg. I assumed by including a world file (jgw) everything would be hunky-dory. I assumed wrong. It was pointed out to me that a projection file (prj) needs to be included, too. That got the raster to display, but brought my poor little server to its knees. Finally, I settled on a GeoTiff. By using jpeg compression within the GeoTiff, I was able to keep the file size reasonable, and GeoServer handled it just fine.
(Thank you @geowolf, @storm72)

Zoom Levels

For the life of me I could not figure out how to zoom in close enough to make my newly loaded raster data useful. I know I had seen many GeoServer/GeoExplorer maps able the zoom in nice and close, but it just wasn’t happening for me. Finally, it was pointed out that changing the default base layer to something other than Google Terrain fixes that problem. Worked like a charm. Why would the Google Terrain base map limit zooming in close? I have no idea. Just one of those things.
(Thank you @manicmapper)

HTML and CSS

After getting these things working, I decided to dress up the web page that holds my GeoExplorer map a little bit. I have Dreamweaver CS4, and I use that to manage a couple of websites, but it’s really much more than I need, especially for a little single page website. I looked around and found Aptana Studio. After using it for an hour or so, I’ve come to really like it. It color codes and formats the HTML and CSS so it’s readable, and I think the preview feature works better than what Dreamweaver has. I might also give Komodo Edit a try, and see how that compares.
(Thank you @mcbride_bryan)

What’s Next?

Probably some more experimenting with the various SLD editors, brushing up on my HTML and CSS skills, and diving into JavaScript, which I’m embarrassed to say, I haven’t really looked at in over 5 years.

What’s it look like?

Here it is as of January 31, 2011:

GeoSandbox 2011-01-31

And a link to the site so you can see it now:

Don’s GeoSandbox

Posted in GIS | Tagged , | 2 Comments

Serving Maps for Free

The Goal

This weekend I started down the road to fulfilling one of my New Year Resolutions: Get some maps online. And by maps online, I don’t mean posting links to KML files, screen shots, or PDF exports from ArcGIS. I mean full-fledged interactive maps viewable by anyone with a browser. On top of that, I had one more goal. I wanted to do this with as little extra cost to me as possible. This is going to be a learning experience for me, not a paying job.

The Map Server

I’ve put a lot of thought into this. It’s something that’s been on my mind for a long time, and I’ve looked into a few options. Choosing a map server was relatively easy. There appear to be three major players in the map server arena. ESRI’s ArcGIS Server, GeoServer, and MapServer. There are others mentioned on Wikipedia’s Web Map Service page, but these are the three that seem to be discussed most often.  ArcGIS Server was out of the question for obvious reasons. Thousand$ of obviou$ reason$. GeoServer and MapServer are two Open Source options, but GeoServer seems to be the more popular option among the open source crowd. I’m not one to pick an option basedGeoServerStatus solely on its popularity, but in this case, popularity also means more discussion in the forums, which in turn means it’s easier to find a solution to any problems that might arise during my own learning process. So, GeoServer it was. You can download GeoServer directly from GeoServer.org, but I decided to go the OpenGeo Suite Community Edition route. What led me to that decision was their great White Paper on The OpenGeo Architecture, which is the most concise, easy to read explanation of how all the pieces fit together that I’ve found anywhere on the web.

The Web Server

The most difficult decision in this whole process was how to host this beast. Yes, I know, I could just install GeoServer on my laptop and play with it there, but I wanted to make these maps public. Isn’t that the whole point of a server, be it geo or otherwise? I have been looking at Amazon Web Services, and signed up for a AWS Free Usage Tier account when they started offering them. The problem with AWS is, it’s just so darn complicated to get IIS7Managersomething set up. I am getting better at finding my way around there, but I still get lost once in a while. Plus, the free accounts require using a Linux operating system. I’ve been experimenting with Ubuntu at home, but I’m much more comfortable working in a Windows world. And Windows usually means paying for cloud computing space (e. g. $58/month at Rackspace). If you’re interested in looking at the AWS option, I highly recommend you take a look at what @spara has done: Install GeoServer on Amazon EC2 without leaving the browser.

The Home Server

I decided to merge this project with another I’ve been thinking about for some time now: setting up a home-based web server. I have an old Dell 600m laptop that’s just taking up space in my office. I have an extra copy of a Windows 7 Home Premium upgrade disc that comes with IIS7. Both were essentially free (to me anyway, your costs may vary). MyHomeWebServerThe only unknown was, would I be able to open this up for public viewing? The main roadblock to setting up a public server using a home internet connection is the use of dynamic IP addressing and port blocking by some (most?) ISPs. I was able to work around both of these obstacles by using DynDNS’s services, and setting my router and software settings appropriately. I wont go into the intricacies here, but if you do a Google search on either of these terms, a wealth of information will flood your browser.

the steps I followed to get things up and running:

  • Wiped the hard drive on the Dell 600m laptop clean
  • Installed the original XP OS, and then the Win 7 Home Premium upgrade
  • Installed all updates through Windows Update
  • Downloaded and installed the OpenGeo Suite
  • Played around with GeoServer and GeoExplorer enough to get a basic map embedded on a web page
  • Got a free Dynamic DNS host name at DynDNS.com
  • Installed DynDNS Updater
  • Modified my router settings to make sure the proper ports were open and forwarding to my laptop server
  • Tested, changed settings, tested, changed settings, tested, tested, tested
  • Scratched head, changed settings, tested, swore a few times, had a beer…
  • Installed and set up Internet Information Services (IIS7)
  • Set up a website using the web page with the embedded map I made with GeoServer/GeoExplorer
  • Went through the whole “testing-scratch head-drink a beer” routine again
  • Celebrated the results

Here’s a snapshot of the results on day 1:

Don's GeoSandbox RC1

And a Link to  what it looks like now:

Don’s GeoSandbox

I’ll try to keep this link up-to-date, but this is all a test, and it is running on a home computer, so don’t be surprised if it’s not available 24/7.
Some of you that have seen this before this post asked the question”Great, but where’s the data?”
Well, that’s coming. Please be patient with me people :-). Baby steps. Baby steps.

Addendum 01/12/2011

I now have a few shapefiles uploaded and viewable on the site. Next step will be to dress up the web page a little bit.

Posted in GIS | Tagged , | 18 Comments