woensdag 16 november 2016

Advanced Tableau

Playing with Tableau and trying some advanced settings:

1.  Adding GPS points to a map; how to. 

2.  Adding shape file to map; how to  and check for maps

maandag 24 oktober 2016


The Economist published a very interesting data journalism piece about the emotions in the presidential debate Trump versus Clinton. "Using facial-analysis software developed by Microsoft, The Economist has sought to quantify Mr Trump and Mrs Clinton’s emotions during some of the most trenchant moments of the debates".
This analysis focuses on facial expression in the debate, the content however is important as well. Using a transcript of the debate published by The Washington Post , I analyzed the debate on keywords, visualized in word clouds and the sentiments related to the words using R.

Here are the word clouds for key words of Clinton and Trump after the 'more tag':

vrijdag 21 oktober 2016


The Nobel prize for literature is awarded to Bob Dylan. How to report about that? From a data journalism perspective there are interesting possibilities. NRC Handelsblad published an info graphics. Interesting, but there are other possibilities using R and Tableau. Here are a few examples. If you are interested in the how-to, follow the more tag.

maandag 26 september 2016

Does Ubuntu shines through the Windows

I tried all possibilities to get Linux/Ubuntu running on a Windows machine. Running in a virtual machine: too slow; using Cygwin, well, not really Ubuntu, still in Windows. Dual booting, yes interesting possibility but you have to reboot if you want to switch.

Since the latest update on Windows10 there is a new solution. Windows creates the possibility to install a Linux layer.  That is: you have a terminal running bash. Here is the howto: http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/ .
Ah you want of course the GUI: Unity for example of Ubuntu. Well you can try, and for me it works. Here is the recipe: http://thehackernews.com/2016/07/ubuntu-gui-windows-10.html
It is not perfect, and sometimes the system crashes. But you can have Ubuntu running together with Windows 10. Now that Microsoft has installed workspaces as well. I run windows in the first and Ubuntu in the second. Finally, the Linux sun shines through the windows.

zondag 11 september 2016

Integrating Google Charts with R

Making charts and graphs in Excel is not a real pleasure; Google sheets works much better esp if you to publish the graphs and charts on line. Here is an example using Worldbank data. However for data analysis R is my first choice. Now I can integrate R with Google charts, using googleVis in R. Here is the handout.
R-Bloggers published the following example for Worldbank data.
Making moving charts is now so easy. Here is my example based on Worldbank data; showing relationship between GDP per cap and Internet access per 100 from 2001 to 2014 for sub sahara african countries.
Data look as follows:
'data.frame': 154 obs. of  4 variables:
 $ country  : Factor w/ 11 levels "Botswana","Ghana",..: 1 1 1 1 1 1 1 1 1 1 ...
 $ year     : int  2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 ...
 $ gdppercap: num  3115 3044 4149 4879 5328 ...
 $ inetperc : num  3.43 3.39 3.35 3.3 3.26 ...
     country        year        gdppercap         inetperc      
 Botswana:14   Min.   :2001   Min.   : 149.4   Min.   : 0.0899  
 Ghana   :14   1st Qu.:2004   1st Qu.: 446.7   1st Qu.: 2.2950  
 Kenya   :14   Median :2008   Median : 930.5   Median : 5.4925  
 Malawi  :14   Mean   :2008   Mean   :1962.4   Mean   : 8.8989  
 Namibia :14   3rd Qu.:2011   3rd Qu.:2919.8   3rd Qu.:11.5000  
 Nigeria :14   Max.   :2014   Max.   :8081.4   Max.   :49.0000  
 (Other) :70 
+                        idvar="country", 
+                        timevar="year")

dinsdag 26 juli 2016


I plan a cycle tour this summer around the city of Venice in Italy: Giro del Veneto. When preparing for the trip on racing bikes we got the GPS tracks. I use my android phone for maps and my friend use a Garmin Edge Touring. I stared studying about how to use GPS tracks, different formats; loading tracks in your phone with Google maps; and finally creating tracks yourself in Google maps and exporting them. The first look into the GPS area reveals: it is a wonderful landscape of different format; Babel at large.  This is what I think you need to know.

woensdag 13 juli 2016

Get your census data from Wazimap

Open Data is a new kid on the journalism block. In 2011 president Obama decided that thousands of data sets of the government could be opened up to scrutiny by the public; (data)journalists, policymakers and businesses. This is the beginning of the open data movement. The new openness aimed at creating more transparency, enhancing efficiency, creating business opportunities.
After the US, European countries joined the open data movement. Let’s take an example. You are filling in your tax form, but you face a problem. There is a service: make a call to the taxation office. But how long would be the waiting time? The service of the taxation office publishes its data: topics of questions, waiting time etc. Analysis of these data could improve efficiency of this service, or diminish mistake in tax forms. Journalists could write a story about wait-times at this service.

maandag 11 juli 2016

CENSUSREPORTER on local host at Myanmar

Publishing open data is an important first step, accessing the data is the second. And that can be difficult. Census reporter  is an important step to access census data. The app or software used to access the census data is open source can be installed on a local machine. I successfully installed it using the this recipe.

woensdag 10 februari 2016


Finally I got it up and and running QGIS map server and the web client. Using a local LAMP stack with Ubuntu 15.10. Here is an example of old data using 'burgemeesters' in Dutch municipalities. The map is working; next some fine tuning.

woensdag 27 januari 2016


Installed a new Linux machine running Kali 2.0, ready for fun.....Do some reading in one of these.

woensdag 20 januari 2016


Mapping tweets used to be a complicated process. I used R, downloading the tweets, then geo coding and mapping. Now there is a nice python program to download, analyze and map the tweets. Here is the installer.
I had no trouble installing in Ubuntu. First install the libraries; then download the the program, unzip and run the python script. Installing twitter API was no problem. Then start download the tweets and put them on a map. Here is een example.

zaterdag 9 januari 2016

Does your secure passwords look like this

I know, passwords, a pain...You have to make them, remember them and they are complicated. Like me, you make a few passwords, safe, but for a professional hacker easy to crack, and you use them several times on different sites. After some time you end up in a situation like this. Your security has dropped to less then 20%; your passwords are chaos.As if you leave your front door wide open with a sign that the key to the vault in vol 1 of Das Kapital in library.