I’m not quite sure about the real practical use of stream graph visualizations, but I admit that they look very nice. In the last week I experimented a little with stream graphs and created a tool which helps finding nice color scales for stream graphs. It allows to define three different scales for each component of the HSL color space. The displayed data set is taken from Piwik Analytics and shows the evolution of visitor countries over the last 16 weeks. Click on the image to enter the live version.

Below you find a small introduction to the different toolbar controls.

Update: I added the possibility to copy the URL of the current settings, which allows to permamently store and share a color scale.

Here are a few quick examples for different settings:

Here are my ideas for the next version of Piwik Maps.

• The map has altogether four views: world, continent¹ and two new country views for provinces and cities.
• In both country views there is one active country. All other countries are displayed in gray like the empty countries in world and continent view (the inactive country borders are missing in the following images, just imagine them for now).
• Since we now have three different zoom levels it is not as easy to provide simple navigation through the views. Starting from the world view, the user can click on active regions to zoom into it². At every time, clicking at the “sea” (the white space) leads to a zoom out to the previous level. After selecting a country the user can switch between province and city view using the icons in the bottom icon bar.
• To improve the navigation between different countries, all inactive countries are labeled with their iso-code (or maybe the full country name). This way the user can easily jump between adjacent countries. Empty countries (= countries with no visitors coming from) are not clickable. The other country labels could be sized relativly to the number of visitors, but this may be to distractive.
• On top of the map the user can see the total number of visits for the active country.

Province view

Now we come to the first new country mode, which displays all sub-regions or provinces of a country. Since maintaining all provinces for nearly 300 countries means a lot of work, this feature may be limited to some countries at first. The tooltip displays the name (again a lot of work for the translators) and the selected metrics. I’m not quite sure about the percentage value, but I tend to relate it to the countries total value (and not the overall total value as in world view), which would mean “4% of the all site visitors of France came from Burgundy”.

City view

The second country view displays all visitors city in the selected country. The concept should be familiar to most web analytics users, so I think it’s a good idea to adapt the bubble layout from GA. Note the change in the icon bar in the following mockup:

If you have any comments or suggestions for the further development of the map widget, feel free to join the discussion in the comment section.

After writing down my latest ideas for better Piwik visualizations, I just couldn’t wait to see them in action. Since the Open Flash Chart library doesn’t support multi-level pie charts I decided to create the visualization from scratch. The initial version (0.1) took me about two hours coding.

In the following image you can compare the hierarchical pie visualization (top) with the current pie visualization (bottom).

I named the plugin UserSettingsExt as it extends the current UserSettings plugin. You can download it from the Piwik Developer Zone to test it on your own Piwik installation.

Changelog

• 0.1 – basic pie chart drawing, label placement.
• 0.2 – added standard piwik tooltips
• 0.3 – added automatic resizing

If you don’t want to miss any updates, you can subscribe to the plugins rss-feed at the Piwik Developer Zone.

Feedback welcome.

In this post I want to present two ideas for improvements to the current Piwik visualizations.

Screen resolution stats

The first idea is about improving the screen resolution charts. Here is an example for the current available visualizations in Piwik:

The current charts (pie,bar) aren’t quite applicable to get an useful overview about the visitors screen resolutions. Both charts are limited to a few big values and combine all other values in the “others” element. This can lead to wrong interpretation of the data, since there might be many different screen sizes. In the example nearly the half of the sites visitors have screen resolutions that arent’ displayed in the charts.

The obvious reason is that pie and bar charts are made for simple, one-dimensional data sets but the number of visits per screen resolution is a perfect representative of two-dimensional data sets. The two dimensions are obviously the screen width and the screen height. One of the simplest yet best visualization type to display two-dimensional data sets is the scatter plot. You can see the difference in no time by looking at the following prototype. In my opinion, this is the only way to make the screen resolution stats usable.

Browser statistics

My second idea is about the browser statistic charts. Again, there is only limited use to the currently provided charts.

The chart shows data of multiple browsers in multiple versions. Each combination of browser and value is treated as a distinct value and hence is shown in a unique color. The main problem of the current visualizations is (again) that the data is treated as one-dimensional. The data is in fact a hierarchical data set: the first layer is the browser layer which contains the sub-layer browser versions. One way of making a visualization more useable is to decrease the complexity, which can be done by only showing the browser level.

The visualization now clearly shows the information that is asked by most of the Piwik users: Which browsers are used by the visitors of my website? Some users might also want to know more details about the different browser versions, so there is a link to an second view that shows all browser versions.

Feedback welcome..

If you’re using Piwik, the open source web analytics tool, then you probably already missed a map visualization of your visitors location, which is a standard feature of all major web analytics tools. As I really like the idea of open source and decentral web analytics and also gained some experiences on map visualizations, I decided to contribute a map plugin to the Piwik community.

Download

You can download the plugin here: UserCountryMap.zip

Update: the current version of my plugin relies on features, that are not available in Piwik 0.6.4 yet, so it doesn’t work now. You have to either wait for the final release of Piwik 1.0, which will be released in August 2010 and will include the map plugin by default, or checkout the current development version via svn.

Features

Interactive world map: The plugin provides a new dashboard widget showing an interactive world map. The countries are colored according to the selected data metric (like number of unique visitors). It is possible to zoom into certain regions to get a closer look at smaller countries. While moving the mouse over countries, tooltips show the exact value and its ratio.

Piwik API integration: The map is fully integrated into the new Piwik report API, so there is no limitation on the displayed data metrics.

Legend: Besides the map there is a small legend that helps to interpret the country colors. This is especially helpful, if you want to print out the map.

Fullscreen: You can set the map into fullscreen mode. This is very handy if you’re working on a smaller screen.

Image export: It is possible to download the current map view as PNG image, just like the other flash-based charts in Piwik. The exported map view will be resized to 1000px width. You can even export high-resolution images by using the image export while in fullscreen mode (accessible via context menu).

Screenshot

This is how the widget looks, after zooming into Europe:

Here is an example for the new fullscreen mode (click image to enlarge):

Current progress / Roadmap

The plugin is ready and waits for the release of Piwik 1.0.
Update: Since Piwik 0.9 the map plugin is one of the core plugins that are also activated by default.

Future versions
There are some more features, which I would like to implement in the future. One is to display vistitors cities as implemented in Google Analytics. This feature needs further improvement on the GeoIP plugin, because by now Piwik does only store the visitors country but not the city. Once this features makes it into Piwik core I’ll update the map plugin.

Donate

If you like the plugin, please support the developer with a small donation via paypal or flattr.

Simplification is a common task in vector-based map visualization. In some situations you don’t want to display a map in its full resolution for reasons like the overall filesize and the rendering time. One major class of simplification algorithms works by removing some vertices without moving the remaining ones. While there are already some good tutorials for implementation of line simplification algorithms online¹, none of them takes into account the special requirements of map data simplifications. This gap will be closed by this post.

At first I want to introduce the used data set. It’s a very basic map of all european countries. The first image shows the output of the map rendering without simplification.

While there are many simplification algorithms out there, this tutorial uses a very simple one. It’s strategy is to remove certain points that are too close to the last kept point. This is not the smartest algorithm ever, but still one of the fastest, because it don’t needs recursion and runs in O(n).

const minDistSq:Number = 100000 * 100000; // radius 100km
var countries:Array, shapes:Array, shape:Array, pt:Object, lastPt:Object, distSq:Number;
 
for each (shapes in countries) {
	for each (shape in shapes) {
		lastPt = shape[0];
		for each (pt in shape) {
			if (pt == shape[0]) continue; // skip first point
			distSq = (lastPt.x * lastPt.x - pt.x * pt.x) + (lastPt.y * lastPt.y - pt.y * pt.y);
			if (distSq < minDistSq) pt.deleted = true;
			else lastPt = pt;
		}
	}
}

This algorithm works perfect on single polygons, but there are some drawbacks when using it for simplication of country outlines. As you can see in the next image, the simplification introduced overlappings and gaps between the countries.

The problem here is that different polygons that share the same boundary were treated seperately. To fix this we need to refine the current map topology (CONTINENT > COUNTRY > SHAPES) in a way, that the “shared vertices” were stored only once.  The algorithm simply iterates over all points and stores them into a global point register. Every new point will then be checked for equality (same x and y coordinates) with any of the previously saved points. If the algorithm finds an equal point, the reference to the duplicate point will be replaced and the custom property inner will be set to true (which means that this point lays on more than one country border). Incidentally this will also decrease the map file size and gives us a classification of  “inner” and “outer” points for free.

When we run the algorithm again, we need to remeber wether the simplification already visited point, to avoid double removal. The results look much better than before, but still need some improvements:

Obviously, there is one case we forgot, which I call three-country-crossings, which means all points that lay on more than two country borders. Removing one of these points leads to triangle shaped holes inside the continent area. To avoid those holes, we got to mark all three-country-crossings as undeletable. We can find three-country-crossings by counting how often we visit a single point during the topology initialization.

Now we are nearly finished. It took me a few days to notice that there is another case, we have to consider. If you compare at the crossover between Spain and France at the Mediterranean Sea side, then you note the tiny cut into the landscape, that is introduced by the simplification.

To avoid those cuts we must identify all vertices that connects two countries and lay on the outside of a continent area. We can do so by comparing the current visited point with the last visited point, and if one lays on two countries and the other only on one, we mark the one on two countries as undeletable. The last picture shows the final output of the simplification:

I hope this tutorial was useful to you, if anyone needs more code examples on this topic, let me know via comment.

This is a small class that I use very often. Its purpose is to simplify the execution of methods after a specific delay. Normally you would have to create a new Timer instance, attach an event listener for the TimerEvent.TIMER event and find a way to store the arguments that you want to pass to the event handler. Using the DelayedExecution class, this whole process goes in one line of code:

function helloWorld(name:String, message:String):void {
	trace( name + ' says "' + message +'"' );
}
 
new DelayedExecution(4000, this, helloWorld, "john", "hello world");
// will result in a 'john says "hello world"' trace after 4 seconds

That’s all. First parameter is the delay in milliseconds, second is the object on which you want to call the method, third is the method itself and any following parameters will be passed to the method. Here you can grep the source of the DelayedExecution class. Feel free to use it for whatever you like.

package net.vis4.utils
{
	import flash.events.Event;
	import flash.events.TimerEvent;
	import flash.utils.Timer;
 
	public class DelayedExecution
	{
		private var
			_obj:Object,
			_func:Function,
			_args:Array;
		/*
		 *  new DelayedExecution(100, Math, round, 4.5);
		 *
		 *  parameters:
		 *  - delay
		 *  - object
		 *  - function
		 *  - parameters
		 */
		public function DelayedExecution(... arguments)
		{
			if (arguments.length < 3) {
				trace('DelayedExecution-Error: missing arguments');
			} else {
				if (arguments[0] is uint || arguments[0] is Number) {
					if (arguments[1] is Object) {
						if (arguments[2] is Function) {
							_obj = arguments[1];
							_func = arguments[2];
							_args = [];
							if (arguments.length > 3) {
								for (var i:uint = 3; i < arguments.length; i++) {
									_args.push(arguments[i]);
								}
							}
							var t:Timer = new Timer(arguments[0] as Number, 1);
							t.addEventListener(TimerEvent.TIMER, exec);
							t.start();
						} else trace('DelayedExecution-Error: third argument should be a function');
					} else trace('DelayedExecution-Error: seconds argument should be an object (this-context of function)');
				} else trace('DelayedExecution-Error: first argument should be a number (milliseconds delay)');
			}
		}
 
		private function exec(e:Event = null):void
		{
			_func.apply(_obj, _args);
		}
	}
}

This is an AIR application I built for an assignment in the course Interactive Systems at the university. This video demonstrates all features like touch guestures and filtering. Since I don’t own a touchscreen, this is only a “fake” touch video. I’d love to try this out on a real touchscreen.

In the last part I gave five improvements to the charts that are shown in the WordPress.com Stats plugin. I ended with the weekly visits chart, so now I conclude with improvements for the monthly visits chart and presenting the idea of putting all the three into one chart.

Improving The Monthly Visits Chart

6. Dealing With Different Month Lengths

It’s easy to understand that comparing monthly visits of january and february is a bit like comparing apples and oranges since all months have different number of days. One way to get over this is to normalize the monthly visits according to the month length. This is simply done by dividing the monthly visits by the number of days in the month. You can see the difference if you compare the ratio of january and february in the normalized chart with the same values in the version above.

7. Avoiding Misinterpretation Of The Last Month

As the chart simply plots the number of visits per month it can cause some misinterpretation of the current month. So, instead of normalizing the months visits by the total number of days we just consider the number of already passed days. This results in an approximated value so it might be a good idea to visually indicate the approximation, for instance by using a dashed line as shown in the following example.

Putting It All Together

8. Integrating All Data Into One Chart

While experimenting with the wordpress statistic charts I got the idea of putting all three charts into one single chart. Therefore the weekly and monthly visit data had be normalized to daily averages as already shown in step 6. I chose a light color for the bar chart that again shows the total number of daily visits. The two line charts were put in front, one showing the average daily visits of a week and the other showing the average daily visits of a month. The lines are drawn in different line width so they can be easily seperated. The color indication for the weekends was replaced by vertical grid lines which helps to identify weeks. The now missing numbers of total visits per week and month were put into tooltips which popup when the user points over the line charts.

9. Smoothing The Line Charts

The purpose of the line charts for average daily visits is to give the user an aproximate image of the visit trends in his blog. To underpin this character of aproximation I smoothed the line charts by replacing the straight lines with cubic bezier splines. This also gives the chart a nicer appearance.

10. Adding Contextual Information To The Chart

As the wordpress stats plugin also stores the number visits for each post and day, the idea came up to integrate information about blog posts in the chart. I decide to introduce another bar chart, that shows one bar for each post. The bar height is calculated using the average daily post visits. The time period considered for this calculation is limited to 45 days to take account of the fact that all posts are losing relevance over time.

To help the user identifing the blog posts I added a small tooltip which appears after hovering over the post bars. Since pointing the mouse to a certain item means that the user shows a special interest in that item, I decided to go one step further and also display the complete post visit statistics. To avoid mix-ups, the other post bars are hidden until the user moves the mouse to a “neutral” part of the chart.

Live-Demo (beta)

You can see the complete visualization in action by looking at the live-demo. If you like to use this visualization for your own blog, please be patient a few days until I fixed all remaining bugs and wrote a tiny documentation for all possible flash variables.

WordPress.com Stats is the most used statistic plugin for wordpress (with over 1.5 mio downloads so far). It offers a very simple way of collecting anonymous visit statistics and has a nice backend that lets you explore the data using line charts and tables. I keep using this plugin since the very beginning of this blog and I must admit that I’m looking at it every day. But in doing so I recently started getting bugged of the charts. This post series shows ten ways for improving them.

Hint: if you want to skip all the boring talk about charts, you can jump directly to the live-demo of the improved wordpress.com stats chart.

Improving The Daily Visits Chart

Here is the original line chart that shows the total number of daily visits:

1. Chosing The Right Chart For The Right Data

Originally, the daily visits are visualized in a line chart. Line charts are good and commonly used to show trends in data over continuous time series. But in the case of our daily visit chart we have a non-continuous time axis. If there were 100 visits on monday and 50 visits on tuesday, we cannot expect 75 visits in the night between monday and tuesday, but this is exactly what the line chart shows us. In cases of discontinuous (or discrete) data, a simple bar chart is the better choice.

2. Showing Indicators For Weekends

Obviously, the number of daily visits follows certain patterns every week. Since there are less people surfing the web on weekends, the number decreases each saturday and sunday. This pattern has a greater influence on the visit  number than any of my posts, so it makes sense to introduce an indicator for weekends. In the following example, I changed the bar colors for saturdays and sundays.

3. Enhancing The Contrast Between Background And Grid

A gradient background from gray to white might be a crucial feature to someone, but for most people this just causes that one cannot perceive the grid. In the next version I changed the color of the background and the grid, which makes the grid a lot more useful.

Improving The Weekly Visits Chart

Now we start looking at the chart for the weekly visits. Again, this is the original line chart that is showed in the wordpress stats plugin.

4. Introducing A More Usable Labeling Of The Time Axis

The time axis is labeled with the week numbers. If you have no calendar next to you, I bet that you don’t know the exact date of the 14th week of 2010, even guessing the right month is difficult. In the next version I labeled the weeks with the date of the first day of the week and put the name of the month below.

5. Using Rounded Values For The Visit Axis

As you can see in the above image, the chart for the weekly visits introduces labels like “344″ or “258″. This might be a little more comfortable for the developer, but it makes the chart less readable. Instead, rounded values like 300 and 400 are more easy to remember and help the viewer interpreting the chart.

In the next part I’ll continue with five more improvements like dealing with different month lengths or adding contextual information to the charts. Do you have any remarks so far? Feel free to post them in the comment section below.