{"id":146,"date":"2012-07-04T05:52:20","date_gmt":"2012-07-04T02:52:20","guid":{"rendered":"http:\/\/ekqvist.goeuropeinfo.com\/rbloggerqvist\/?p=146"},"modified":"2012-07-04T05:52:20","modified_gmt":"2012-07-04T02:52:20","slug":"r-and-eurostat-bulk-data-heatmap-example","status":"publish","type":"post","link":"https:\/\/science.ekqvist.fi\/blogi\/r-and-data-visualization\/r-and-eurostat-bulk-data-heatmap-example\/","title":{"rendered":"R and Eurostat bulk data (Heatmap example)"},"content":{"rendered":"<h1>R and Eurostat bulk data<\/h1>\n<p>In this Exercise I am testing <a href=\"http:\/\/epp.eurostat.ec.europa.eu\/NavTree_prod\/everybody\/BulkDownloadListing\" target=\"_blank\">Eurostat bulk data source<\/a> and plot these data into Heatmap. Let&#8217;s try with this data:<br \/>\n&#8220;Harmonised unemployment rates (%) &#8211; monthly data (ei_lmhr_m)&#8221;<\/p>\n<p><a href=\"http:\/\/science.ekqvist.fi\/blogi\/wp-content\/uploads\/2012\/07\/Harmonised-unemployment-rates-percent-monthly-data.jpg\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-147\" title=\"Harmonised unemployment rates percent monthly data\" src=\"http:\/\/science.ekqvist.fi\/blogi\/wp-content\/uploads\/2012\/07\/Harmonised-unemployment-rates-percent-monthly-data.jpg\" alt=\"\" width=\"480\" height=\"480\" srcset=\"https:\/\/science.ekqvist.fi\/blogi\/wp-content\/uploads\/2012\/07\/Harmonised-unemployment-rates-percent-monthly-data.jpg 480w, https:\/\/science.ekqvist.fi\/blogi\/wp-content\/uploads\/2012\/07\/Harmonised-unemployment-rates-percent-monthly-data-150x150.jpg 150w, https:\/\/science.ekqvist.fi\/blogi\/wp-content\/uploads\/2012\/07\/Harmonised-unemployment-rates-percent-monthly-data-300x300.jpg 300w\" sizes=\"(max-width: 480px) 100vw, 480px\" \/><\/a><\/p>\n<p>#<a href=\"http:\/\/epp.eurostat.ec.europa.eu\/portal\/page\/portal\/statistics\/bulk_download\" target=\"_blank\">You will also find Eurostat Data source from here<\/a>:<br \/>\nhttp:\/\/epp.eurostat.ec.europa.eu\/portal\/page\/portal\/statistics\/bulk_download<\/p>\n<p><code><br \/>\n#I will automate this data downloading and extracting, but just now this is semiautomatic<br \/>\n# create download directory and set it<br \/>\n.exdir = 'c:\/data\/tmp2' # put there your own data folder<br \/>\ndir.create(.exdir)<br \/>\n.file = file.path(.exdir, 'ei_lmhr_m.tsv.gz') # change this<br \/>\n<\/code><code><br \/>\n# download file<br \/>\nurl = 'http:\/\/epp.eurostat.ec.europa.eu\/NavTree_prod\/everybody\/BulkDownloadListing?sort=1&amp;downfile=data<br \/>\n<\/code><code><br \/>\n%2Fei_lmhr_m.tsv.gz'<br \/>\ndownload.file(url, .file)<br \/>\n<\/code><code><br \/>\n# untar it (Note: I do not know why I got error message: Error in getOct(block, 100, 8) : invalid octal digit)<br \/>\nuntar(.file, compressed = 'gzip', exdir = path.expand(.exdir))<br \/>\n<\/code><code><br \/>\n# Argh...something going wrong with this step, so I have to manipulate just downloaded data. First I remove comma<br \/>\n<\/code><code><br \/>\n# from very first variables and etc... I always use Notetab light as a Text editor in this kind of task.<br \/>\n<\/code><code><br \/>\n# Reading file into R. Please refer here your own data folder...<br \/>\ninput &lt;- read.table(\"c:\/data\/tmp2\/ei_lmhr_m.tsv\", header=TRUE, sep=\"\\t\", na.strings=\":\", dec=\".\", strip.white=TRUE)<br \/>\n<\/code><code><br \/>\n#just checking<br \/>\nhead(input)<br \/>\n<\/code><code><br \/>\n<\/code><code><br \/>\n# LM-UN-T-TOT = Unemployment rate according to ILO definition - Total rate<\/code><code><br \/>\n# NSA = not seasonally adjusted<br \/>\ninput&lt;- input[which(input$indic==\"LM-UN-T-TOT\"),]<br \/>\ninput&lt;- input[which(input$s_adj==\"NSA\"),]<br \/>\n<\/code><code><br \/>\n#giving appropriate names in to the heatmap (without this manouver there will be only row id)<br \/>\nrow.names(input) &lt;- input$geo.time<br \/>\n<\/code><code><br \/>\n#just checking<br \/>\nhead(input)<br \/>\n<\/code><code><br \/>\n#Column selection. We will get data between time period 05\/2008 - 05\/2012<br \/>\n<\/code><code><br \/>\ninput2 &lt;- input[,5:53]<br \/>\n<\/code><code><br \/>\n# data frame must change into data matrix\u00a0 to produce heatmap.<br \/>\ninput_matrix &lt;- data.matrix(input2)<br \/>\n<\/code><code><br \/>\n#heatmap is almost here<br \/>\ninput_heatmap &lt;- heatmap(input_matrix, Rowv=NA, Colv=NA, col = heat.colors(256), scale=\"column\", margins=c<br \/>\n<\/code><code><br \/>\n(5,10), xlab = \"Harmonised unemployment rates (%) - monthly data\", ylab= \"Country or Area\")<br \/>\n<\/code><code><br \/>\n<\/code><code><br \/>\n#saving heatmap into folder<br \/>\njpeg(\"G:\/data\/home\/2012\/marko\/blogi_rbloggerqvist\/data\/eurostat\/Harmonised unemployment rates percent<br \/>\n<\/code><code><br \/>\nmonthly data.jpg\")<br \/>\ninput_heatmap &lt;- heatmap(input_matrix, Rowv=NA, Colv=NA, col = heat.colors(256), scale=\"column\", margins=c<br \/>\n<\/code><code><br \/>\n(5,10), xlab = \"Harmonised unemployment rates (%) - monthly data\", ylab= \"Country or Area\")<br \/>\ndev.off()<br \/>\n<\/code><code><br \/>\nHave fun,<br \/>\nMarko<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"<p>R and Eurostat bulk data In this Exercise I am testing Eurostat bulk data source and plot these data into Heatmap. Let&#8217;s try with this data: &#8220;Harmonised unemployment rates (%) &#8211; monthly data (ei_lmhr_m)&#8221; #You will also find Eurostat Data source from here: http:\/\/epp.eurostat.ec.europa.eu\/portal\/page\/portal\/statistics\/bulk_download #I will automate this data downloading and extracting, but just now <a class=\"read-more-excerpt\" href=\"https:\/\/science.ekqvist.fi\/blogi\/r-and-data-visualization\/r-and-eurostat-bulk-data-heatmap-example\/\">[&#8230;] Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"gallery","meta":[],"categories":[19,22,23],"tags":[37,38,48],"_links":{"self":[{"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/posts\/146"}],"collection":[{"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/comments?post=146"}],"version-history":[{"count":0,"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/posts\/146\/revisions"}],"wp:attachment":[{"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/media?parent=146"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/categories?post=146"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/science.ekqvist.fi\/blogi\/wp-json\/wp\/v2\/tags?post=146"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}