Thursday, 18 December 2014

A note about .net extension methods

If you've ever programmed in C#, you've probably encountered this one awesome feature called extension methods that allows you to add new methods to existing types without modifying the original type (in reality, this feature is nothing more than syntactic sugar for invoking utility methods in a natural object-oriented manner. Still, being able to call utility methods in such a manner is much better and is partly the reason why the various LINQ APIs are so elegant to use)

One thing that people may overlook when writing their extension methods is the namespace in which the extension method class is defined in.

Simply put, if you are going to write extensions methods for a class/interface/enum, you should define the extension method class in the same namespace as the class/interface/enum that you are "extending"

The reason for this is simple: It improves discover-ability of these extension methods. Any time you bring in a namespace for a given class/interface/enum, the extension methods (being in the same namespace) will also be brought in automatically.

If these extension methods were in a different namespace, you'd have to manually seek out the namespace where these extension methods live in, which ranges from inconvenient to annoying. I've had to ILSpy many assemblies in my time just to find the proper namespace of this one extension method that the documentation keeps referring to, all because the extension methods lived in a separate namespace from the "extendee" class/interface/enum.

Something to think about before you write your next extension method.


Thursday, 20 November 2014

MapGuide tidbits: Log monitoring with log.io

If the previous MapGuide log monitoring solution don't cut it for you (especially on Linux), here's another one you can try.

log.io is a real-time log monitoring solution that runs in your web browser. It is powered by node.js and socket.io.

Just a note before we dive in, this post is geared towards Linux-specific installations of MapGuide. This may or may not work for Windows. The log.io install instructions assume Linux, so we're rolling with that.

To install log.io you will first have to install node.js, there's a bajillion different links out there on how to install node.js, so here's one for:


Once node is installed, you can install the log.io package via npm (on Ubuntu you'll have to elevate that command with sudo):

npm install -g log.io --user "[username that will run log.io]"

Then start the log.io server.

log.io-server

Then create a harvester.conf in your ~/.log.io/ directory that defines what log files to monitor. Here's a basic example configured for MapGuide and Apache log monitoring.

exports.config = {
    nodeName: "mapguide_server",
    logStreams: {
      apache: [
        "/usr/local/mapguideopensource-2.6.0/webserverextensions/apache2/logs/access_log",
        "/usr/local/mapguideopensource-2.6.0/webserverextensions/apache2/logs/error_log"
      ],
      mapguide_access: [
        "/usr/local/mapguideopensource-2.6.0/server/Logs/Access.log"
      ],
      mapguide_error: [
        "/usr/local/mapguideopensource-2.6.0/server/Logs/Error.log"
      ]
    },
    server: {
      host: '0.0.0.0',
      port: 28777
    }
  } 

Now start the log harvester.

log.io-harvester

Now browse to http://localhost:28778 and watch your MapGuide log files in real time.


NOTE: The above install instructions were pilfered from the log.io website itself, however I found the global install option to be problematic on my Ubuntu test VM. For some reason it always insists on building the native modules as the "root" user and not the user I designated from the npm install command. So I went for a local install instead, which means the following commands have been changed to the following:

  • Installing log.io: npm install log.io
  • Running log.io server: ~/node_modules/log.io/bin/log.io-server
  • Running log.io harvester: ~/node_modules/log.io/bin/log.io-harvester

Friday, 7 November 2014

Announcing: mapguide-rest 0.11.2

Here's another release of mapguide-rest. There was some issues that cropped up after pushing the original 0.11 release that necessitated 2 emergency point releases to fix. As a result, 0.11.2 is the official release.

Here's what's new in this release.

KML Service Support

mapguide-rest now provides RESTful routes that wrap the existing KML Service APIs in MapGuide:

  • /library/{resourcePath}.MapDefinition/kml
  • /library/{resourcePath}.LayerDefinition/kml
  • /library/{resourcePath}.LayerDefinition/kmlfeatures
These routes respectively wrap or replicate:
  • GetMapKml (re-implemented in PHP to write the correct RESTful URLs for NetworkLink) elements)
  • GetLayerKml (re-implemented in PHP to write the correct RESTful URLs for NetworkLink) 
  • GetFeaturesKml (wraps existing API)

Pagination support for feature data routes

These following routes that return feature data now support pagination:

  • /library/{resourcePath}.FeatureSource/features/{schema}/{class}
  • /library/{resourcePath}.FeatureSource/features.{type}/{schema}/{class}
  • /library/{resourcePath}.LayerDefinition/features
  • /library/{resourcePath}.LayerDefinition/features.{type}
  • /session/{sessionID}/{mapName}.Selection/features/{layerName}
  • /session/{sessionID}/{mapName}.Selection/features.{type}/{layerName}

You can paginate the results by passing two additional parameters:

  • pagesize - The number of features to return per "page"
  • page - The page "number" to return
Pagination will work for any supported representation except for CZML. The implementation of CZML output is not compatible with our pagination and errors will be thrown if attempting to pass in pagination parameters for CZML output.

The primary beneficiary of pagination support is ...


HTML representation for feature data

We've added HTML representations for feature data routes:
  • /library/{resourcePath}.FeatureSource/features.{type}/{schema}/{class}
  • /library/{resourcePath}.LayerDefinition/features.{type}
  • /session/{sessionID}/{mapName}.Selection/features.{type}/{layerName}
Where {type} can now be html in addition to the existing supported representations.

For example, a html representation of a Layer Definition would look like this



Apply pagination, and you now have a page-able view of your feature data


The real payoff for implementing this feature is that you also get this support for selected features. For selected features, we support changing the display orientation in the HTML representation as well.


Tweak the page size and orientation parameters for this route and you have what is now effectively a selected feature property palette for your custom map viewer for free, as demonstrated in a new sample that is bundled with this release.


The palette in this sample is just an iframe to this new HTML representation URL.

Improved HTML repository view

The HTML repository representation has been cleaned up

You'll notice each resource no longer show all actionable links. That's because these have been moved to a separate page that is loaded when you click on the resource link.

Clicking on any actionable link will show the result in the space below


We've also added more actionable links for the following resource types:

  • Feature Sources: View features as HTML
  • Map Definitions: Export as KML

Various other HTML representations have also cleaned up css.

Localization-ready

mapguide-rest is now localization ready. If you want to translate mapguide-rest to your language of choice, you just have to do the following:

  • Translate and copy the strings in app\res\lang\en.php to app\res\lang\{locale}.php. A small caveat with translating these strings is that you will have to translate the content in a way that respects the ordering of placeholder tokens as we use sprintf to fetch and format localizable strings.
  • Translate and copy the system templates under app\res\templates\en to app\res\templates\{locale}
  • Translate and copy the XSLT templates under app\res\xsl\en to app\res\xsl\{locale}

Once you have translated these files, simply change the Locale property to {locale} in app\config.php and mapguide-rest will use the translated strings and templates

Other changes/fixes

  • Client agent and IP addresses are now recorded when creating a site connection, ensuring all API access from mapguide-rest will be properly logged in the server's access.log
  • Fix layered PDF plotting for maps with base layer groups
  • Added various bits of missing REST API documentation
  • Error messages will now be returned in the appropriate requested format where applicable (eg. If an error occurs when making a request to a URL expecting an XML response, the error will be returned as XML). If no applicable request format can be determined, then the default error response will be in HTML.

Download

Monday, 20 October 2014

MapGuide tidbits: MapGuide Server daemon doesn't start after reboot

This one will be short and sweet.

If you have rebooted your Linux server, and for some reason you can no longer start the MapGuide Server as a daemon. You should check that the /var/lock/mgserver directory exists and create it if it doesn't.

The mgserver process will try to create and lock a file in this directory and will bail out if it can't. This directory is cleared when the Linux server is restarted (at least in my observations). None of the wrapper scripts (mgserver.sh or mgserverd.sh) actually check if the directory exists, so they blindly proceed as though this directory existed.

We'll patch the mgserverd.sh script to create this directory if it doesn't exist before running the mgserver daemon. In the meantime, you can edit the mgserverd.sh file in the MapGuide Linux installation yourself to create the /var/lock/mgserver directory before running the mgserver process.

Thursday, 25 September 2014

Docker-izing MapGuide

There was a motivation behind this public service announcement.

It was part of my investigations into being able to deploy MapGuide as a Docker container.

What is Docker you may ask? Have a read of their introduction page.

TL;DR? Docker is a tool that allows you to run and deploy software inside virtualized software containers. This is not a heavyweight VM like VirtualBox and their ilk. This is something that leverages existing features in the Linux Kernel to provide "lightweight" VMs with very low overhead.

A docker-ized application gives us various benefits:
  • Your application and its dependencies are all self-contained and will not interfere with anything outside of its container and vice versa.
  • Deployment is dead simple. No figuring out what pre-requisites that have to be installed. They'll all be part of the docker image that you can pull down with a single command.
  • Your application can run in host environments that the application was not originally compiled/tested for. Build once, run anywhere (where docker is installed :))
There's many more benefits than what's listed here. So you can probably see where a docker-ized MapGuide would really be useful from a developer and administrator perspective.

But there is a catch to all this goodness. Firstly you need a Linux distribution that uses Linux Kernel v3.8 or newer as that contains the required OS virtualization features needed for Docker to work.

Secondly, you need to run a 64-bit Linux distribution as that is what Docker only supports. In order to run MapGuide within a Docker container, we need a way to run a 32-bit MapGuide within a 64-bit Linux environment as Docker can only be run from within a 64-bit Linux host and as previously mentioned, we still don't have a functional 64-bit Linux build of MapGuide yet :( So the workaround is to install the required 32-bit packages, which you can find in my previous post.

So after applying all of this newly acquired knowledge, allow me to introduce my first Docker image for MapGuide.

This is a CentOS 6 base image that has the following software and packages pre-installed:
From this base image, you can build your own docker image that installs your own MapGuide data, applications and configurations and deploy/run that image as a docker container.

Here's a basic example of getting a docker-ized MapGuide up and running with mapguide-rest and some sample data packages using 64-bit Ubuntu 14.04 using the above docker base image:

Firstly, we install the docker package like so

sudo apt-get install docker.io

Then make a new directory and create a file named Dockerfile


Put the following text into the Dockerfile. The comments should be self-explanatory. 


# This image is based from the MapGuide docker base image
FROM jumpinjackie/mapguide-base
# Install additional packages, load your MapGuide applications and data, etc.
# Download Sheboygan dataset
RUN wget -P /usr/local/mapguideopensource-2.6.0/server/Packages http://download.osgeo.org/mapguide/releases/2.6.0/Release/Sheboygan.mgp
# Download Melbourne dataset
RUN wget -P /usr/local/mapguideopensource-2.6.0/server/Packages https://github.com/jumpinjackie/mapguide-sample-melbourne/releases/download/v0.2/Melbourne.mgp
# Download mapguide-rest 0.10
RUN wget https://github.com/jumpinjackie/mapguide-rest/releases/download/v0.10-pre/mapguide-rest-0.10.zip
# Set up install location for mapguide-rest
RUN mkdir -p /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest
# Extract mapguide-rest
RUN unzip mapguide-rest-0.10.zip -d /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest
# Fix up permissions of the cache directory (for tiles and smarty templates)
RUN chown daemon:daemon /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest/cache
# Expose the web server port to the world outside the container. The default port is 8008
EXPOSE 8008
# Run supervisor, that will start the MapGuide Server and Apache httpd server
CMD ["/usr/bin/supervisord"]


Now save the Dockerfile, and run the following command to build the docker image from that file

sudo docker build -t my-mapguide-app .

This command will download the mapguide-base docker image which will take a few moments depending on your download speed. This is only downloaded once and will remain until you explicitly remove this base image.

my-mapguide-app will be the name of the docker image, which you'll need to reference when you will run a container from it, which is what we will do next. But before we do that, we should list our docker images and see if our new image is there.

sudo docker images



Now that we have confirmation that our image has been created, we can create a container from it like so.

sudo docker -d --name mapguide -t my-mapguide-app

The -d switch indicates that this container will run in the background. The --name switch assigns the name mapguide to this container so we don't have to remember a long UUID for referencing this container in future operations, which is what is outputted when the command succeeds.


Finally the -t switch indicates that we want to create a container from the my-mapguide-app image that we just created.

Now that we have started a container, we now have:

  • MapGuide Server running
  • The Sheboygan and Melbourne sample data packages downloaded to the Packages directory of the MapGuide Server installation directory
  • The mapguide-rest extension installed
  • Apache HTTP Server running on port 8008 which is exposed
We now just need to know what IP address has been assigned to this container. To do that, we can run the following command

sudo docker inspect mapguide | grep IPAddress


Now that we have an IP address, we can see if things are in order by firing up the Site Administrator


If we login and go to the Manage Packages page, we can see the packages we've downloaded as part of building our my-mapguide-app Docker image are there.


Now load these packages, and go to the mapguide-rest sample apps landing page just to see that the extension was installed.


If the samples on this landing page work, you have just verified that your docker-ized MapGuide container is now fully operational.

So that's a little run-through of how to get MapGuide running in a Docker container. Now the other thing about Docker that is really awesome is that the big players in Cloud Computing already (or starting to) support Docker containers as a PaaS deployment option.

Can you say MapGuide on the cloud?


Now just to clarify, MapGuide on the cloud via IaaS is already relatively easy. I, the AWS noob easily got a MapGuide demo site on Amazon EC2 set up over the GovHack weekend, which stayed up long enough to impress the judges to win some awards.

However the IaaS approach to deployment puts a burden on you to maintain the actual infrastructure (VMs, etc). As a case in point, our GovHack demo site has been taken down because I butchered my Amazon EC2 instance due to a combination of failure to act on an important email alert on this issue from Amazon and my general AWS noob-ery.

The PaaS approach lets you focus solely on deploying the applications, without having to worry about the underlying infrastructure. MapGuide as a Docker container enables the possibility of using this cloud deployment option.

Hopefully this post has sold you on the power and potential of Docker and how having MapGuide as a docker container may open up deployment and development scenarios that were previously not possible or took a lot of manual effort.

As for that Docker base image I linked to, that is just the tip of the iceberg. I'm just getting my feet wet with Docker and foresee many future blog posts on this topic as I learn more about what Docker can and can't do.

I can see the possibility of having different base images in the future:
  • Individual Server and Web Tier Docker images. Imagine how easy it would be to set up a load-balanced cluster via Docker containers? I'd have to grok how these containers communicate first, but the possibilities are very interesting.
  • Web Tier images tailored for the PHP or Java installation profiles.
  • Many others?
I also wouldn't comfortably say that the above docker image is currently production-ready. There's questions that need to be answered:
  • How do we handle the case of a mgserver or httpd process falling over on a background docker container?
  • How can we easily access and manage log files in a docker container?
  • How can we perform repository backup/restoration operations within a docker container?
  • How do we perform $MAPGUIDE_SERVER_ADMIN_TASK within the docker container?
If you have an idea on how these questions can be answered, I invite you to help improve the design of this docker image or to enlighten me on the comments below.

MapGuide tidbits: Strange fonts on Linux

Does your MapGuide Linux installation render labels that look like this?


Instead of this?


This is due to MapGuide not being able to correctly locate the required font in question (Verdana). To fix this, you need to install the appropriate font package that provides the missing font. In the case of verdana, that package is msttcorefonts.

On Ubuntu: Simply install the msttcorefonts package

On CentOS: Install the following packages:

  • curl
  • xorg-x11-font-utils
  • fontconfig
And then install the following RPM packages with the rpm command
  • rpm -i http://pkgs.repoforge.org/cabextract/cabextract-1.4-1.el6.rf.i686.rpm
  • rpm -i https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm
Now you can verify your font in question is install by running the fc-list command and looking for the name of your font in the output

fc-list | grep Verdana

Now even after this, MapGuide may still not properly locate this font even if it is installed.

If this is still the case, copy the physical font files to the directory of the mgserver executable (eg. /usr/local/mapguideopensource-2.6.0/server/bin) and restart the MapGuide Server.

In the case of msttcorefonts, the physical font files are found in:
  • On Ubuntu: /usr/share/fonts/truetype/msttcorefonts
  • On CentOS: /usr/share/fonts/msttcore

Tuesday, 23 September 2014

MapGuide tidbits: Running 32-bit MapGuide on 64-bit Linux

We still don't have our mythical 64-bit build of MapGuide on Linux yet. So in the meantime, should you want to run the 32-bit CentOS or Ubuntu builds of MapGuide in their respective 64-bit versions, here's the packages you will need to have installed beforehand.

On 64-bit Ubuntu: Just install the ia32-libs package

On 64-bit CentOS: You will need to install the following packages:
  • glibc.i686 
  • libstdc++.i686 
  • expat.i686 
  • libcurl.i686 
  • pcre.i686 
  • libxslt.i686 
  • libpng.i686
This will satisfy the dependencies required by your 32-bit MapGuide, its bundled Apache HTTPD Server and PHP.

The bundled Tomcat and Java wrapper API has not been tested under this environment at the time this post was published, so to take a logical shot in the dark, you probably need to just install the respective 32-bit JVM package for the Tomcat and Java wrapper API to work. If it turns out I'm wrong about that, please do correct me in the comments below.