Skip to content

Posts 

Marble's Secrets Part III: The Earth in a Download

Tuesday, 12 February 2008
Today we'll finish our first trilogy about Marble Desktop Globe: Part I was a Do-It-Yourself course about creating maps for Marble. We've seen how Marble manages to even display features such as aerial photos or OpenStreetMap. Part II showed how Marble paints the different map layers. We've also seen why Marble only needs very little disk space and memory for its default map - making it an ideal choice for solutions that involve little hardware resources (like the Asus EeePC and the OLPC). In Part III we'll look beyond Marble's offline mode: We'll get to know how Marble fetches its data from the internet. Caption: Left: Creating "KWorldClock 2" in 5 minutes using Marble's Qt Designer Widget Plugin. Note that this is just a proof that this would work today! - Actually Henry de Valence is working on a KWorldClock plasmoid already. Right: Marble showing off a real-time cloud map and Sun Shading with the "Earth At Night" map blended in. By data we mostly refer to texture data. As of version 0.5.x (KDE 4.0) we only support a single texture layer. However this is about to change with the next version of Marble: David Roberts has added support for Sun Shading in Marble. If this wasn't exciting enough he also added a way to have the "Earth at Night" map blended onto the dark side of the earth. This includes a pretty nice Terminator line. On top of that he already added internal support for having real-time clouds as another layer! See the screenshot based on a current build of the Qt-version of Marble pre-0.6-SVN. A really big mosaic of tiles ... Actually I've met a lot of people recently who weren't aware of the fact that Marble downloads further texture data from the internet while browsing the map. On one hand the reason for this misconception is maybe the fact that we only provide very little additional online data right now (about 400 MB). On the other hand the map download is done automatically and very seamlessly in the background, so people will hardly notice. Andrew Manson created a nice YouTube video while he finished his GSoC Project "GPS support for Marble". During the first 1.5 minutes Andrew demonstrates what could be a summary of the "Marble Secrets Part II" hands-on paragraph: the video shows how you can hide and show layers using the legend tab. Watch closely at 1:35 where Andrew zooms into the "Earth at Night" Map: you can see how the map will suddenly show a lot more detail as new data has arrived on the computer in the background. Keep watching and you'll see the same happening with the "Satellite View" map. From Part I we know that internally the current map gets provided at different sizes - each size at different resolutions. These different map "levels" are split into 675x675 pixel tiles each. Depending on the zoom level only those tiles that are actually visible will get loaded into the physical memory ("RAM") of our computer. This mechanism keeps physical memory consumption low. For each tile Marble will check first whether it's available in the physical memory's Tile Cache (yes, you were about to suggest this feature, but we got it implemented already!). If it's not available in the "Tile Cache" Marble will try to load it from the local hard disc. In case it's not there Marble will schedule the tile for download in Marble's very own HttpDownloadManager. But wait - what should we display in the meantime while there is no matching tile available? After all due to the way Marble works we can't simply "omit" the missing tile! As a cheap but smart workaround Marble just scales up the tile at the biggest zoom level that is locally available for this particular place. It will then just cut out the area that is needed and will use this handcrafted DIY-tile as a temporary replacement until the "real" data arrives. In the meantime the HttpDownloadManager will contact the server and will look for the tile in question (e.g. it might look for this one ) . It's only convenient that on the server all the tiles get stored in the very same directory hierarchy as on the local hard disc: {marble-data-base-url}/maps/earth/{theme}/{level}/{row}/{row}_{column}.jpg So Marble will fetch the tile from the server and will store it locally on the hard disc - again sticking to the given tile directory hierarchy. Once it got stored locally Marble will update the temporary data of the texture tile to match the "real" data. This mechanism works pretty well already. The HttpDownloadManager is also used to download the Wikipedia webpages that get displayed if you left-click on city labels or symbols. If you're a real GIS person you might wonder whether it's possible to just replace this HttpDownloadManager with something "more sane" - like an enterprise database that has this data stored already. Yes, a suitable solution might be available in marble/src/lib/MarbleWidget.{cpp,h} where you'll find a method called: void MarbleWidget::setDownloadManager( HttpDownloadManager *downloadManager ); (soon to be changed to "AbstractTileProvider"). Just replace the HttpDownloadManager pointer with a pointer to your very own custom TileProvider class. Get Hot New Maps ... But what if you don't just want to zoom into maps that exist on your hard disc already. What if you want new maps on your computer? DXS to rescue! DXS is the successor of "Get Hot New Stuff (GHNS)". It's a web service that allows applications to download and install data from the Internet in one single click. So if you have the KDE 4 version of Marble installed then you can simply choose the "File" menu and pick "Get New Data ..." there. A dialog will appear that will allow you to download further completely new maps. Usually the maps that get downloaded via DXS only consist of a single tarball. The tarball has a size of a few Megabytes, contains a .dgml-file and might either have an installmap or pre-tiled data. The beauty is that once the .dgml file and a few base tiles are in place Marble can start again to look for further tiles on the server. But why doesn't Marble just download all the tiles from the server in a single download? Well, Blue Marble Next Generation is the most comprehensive free satellite map that covers the whole earth at a resolution of 500 meters per pixel. And this map takes about 350 Megabytes on the server already. With increasing resolution the size occupied on the server will grow exponentially. Even if you cover just the continents at a resolution of up to 30 meters per pixel ( Like NASA's NLT LandSat map does ) then your satellite map will occupy almost 200 Gigabytes on the server! So it's pretty obvious that for an application like Marble it's the best solution to just download the pieces that are actually needed! We plan to provide higher resolution maps on the server in the future. Given that we can't just change the maps easily once we start to provide them, this requires careful planning, so please be patient. If you're using the Qt-only version of Marble then you're out of luck as it doesn't have the DXS feature built-in (It does have the automatic HttpDownloadManager though). You're then on your own and you need to find and install the maps manually. Marble Junior Jobs Today we have several exciting opportunities to let your name enter the CREDITS section in Marble. The "EASY Category" might only take you as little as an hour to complete! To make sure that you won't waste time on a Junior Job ("JJ") while somebody else might already be about to submit a patch you can claim a JJ. Just tell us in the comments section that you have just started to work on it. Please be aware that once you have started time is running up! For reasons of fairness you should send us a preliminary patch within the following 8 hours. This will make your claim permanent for the next few days. Of course the preliminary patch doesn't need to fully solve the problem but it should at least tackle a small part of it. If you don't send us the patch within 8 hours then the JJ will be up for grabs again. I hope this takes away the "uncertainty" whether it's worth to start or not ;-) First however you need to spend 10 minutes to get Marble from the sources and to compile it. You need to check out Marble "trunk". Once that is done you can take the plunge: Category EASY: The Sun Shading feature is only available via a single menu entry in the Qt-Only version of Marble. Please add such a menu entry to the KDE 4 version of Marble: just compile Marble for KDE 4. You need to apply a few changes to marble/src/marble_part.cpp and marble/src/marbleui.rc. Look at the existing menu entries in those files to see how things get implemented. As a reference for the menu name and its position have a look at marble/src/QtMainWindow.cpp. Category EASY: Marble doesn't save or restore the Sun Shading settings. Create a patch that solves this problem. Have a look marble_part.cpp (if you want to implement it for the KDE version of Marble) or have a look at QtMainWindow.cpp (if you want to implement it for the Qt-Only version) to see how it works. Category MEDIUM:: Currently Marble only tries to fetch tiles once. If the server doesn't reply Marble won't retry until Marble gets restarted. Find a solution that will reschedule the tile up to 3x in case of failure. Between the attempts Marble should wait 1 min. before the job gets rescheduled. Relevant classes for this job are marble/src/lib/HttpDownloadManager.{cpp;h} and marble/src/lib/HttpFetchFile.{cpp;h} Category MEDIUM:: This job will take a bit longer: Create a working dialog that will show the Tile Download Status in Marble's queue. The dialog should only be available by starting Marble with a special command line option or through a key stroke. Category HARD:: Marble downloads the tiles and saves them into the user's .marble directory. Until now Marble will never start to delete any local tiles. So the contents in .marble will grow until all tiles got fetched or until the HDD is full. Come up with a solution that will fix this problem. The solution should still preserve the tile directory structure and should work similar to QPixmapCache. Just that in this case the configurable cache is the hard disc. For help or questions you can join us on IRC ( irc.kde.org, #kde-edu ) or send a mail to our mailing list.

Nepomuk Appendix A - RDF for Dummies in a Nutshell

Tuesday, 12 February 2008
In my previous posts I used some terms that probably need explaining. The following descriptions should not be used as basis for any exam and may very well scare some academic semantic web professionals, but they get me through the day. And I think they are sufficient to understand most of what is going on with Nepomuk data in KDE. Read More

Fetch, Nepomuk, fetch!

Monday, 11 February 2008
Search - a very important topic when it comes to data in general. The same is true for metadata and all that is Nepomuk. I blogged about the virtual folders idea for KMail which will be realized through Nepomuk. But before that there is the "simple" desktop search. We know it from systems like Beagle or Strigi. With Nepomuk, however, a lot more is possible. We are just getting started. Read More

Marble's Secrets Part II - Walking In The Shoes Of Slartibartfast ...

Sunday, 10 February 2008
In Part I we have seen how easy it is to create your own maps for Marble. We've also seen how this works down to the zoom level of aerial photos or OpenStreetMap. Part II will show how Marble manages to provide the biggest bang for the byte when it comes to providing map data. It outlines how we managed to get the default map squeezed into the 10MB package that the Marble-Qt version gets shipped with. We basically show ... Read More

openSUSE and Tracking 4.1 Development

Sunday, 10 February 2008
First two mentions from an exciting openSUSE week: the new openSUSE evan..uhm community manager has been finally disclosed, it's Joe "Zonker" Brockmeier. And openSUSE 11.0 Alpha 2 has been released with which we put KDE 4.0 to the test as default desktop. To keep up with openSUSE happenings occasionally I recommend openSUSE Weekly News btw... Read More

Akonadi KResource plugins

Saturday, 9 February 2008
I just moved the two Akonadi KResource "bridges", i.e. implementations of the KResource plugins for contacts and calendars based on Akonadi (KABC::Resource and KCal::ResourceCalendar respectively), to kdepim/kresources. If you are intersted in testing them, you need an Akonadi setup with at least one resource for the respective data type, e.g. an Akonadi vCard resource for testing the kabc plugin. Run akonadiconsole to check and/or add such resources and it would probably be wise not use valuable data :) Read More

Marble's Secrets Part I: Behind the Scenes of Marble...

Saturday, 9 February 2008
This is the first part of a new series about Marble. I'll try to address a few frequently asked questions as well as the current status of the development. So stay tuned. If you've ever followed KDE 4 development then you've probably heard about Marble. Marble is a virtual globe which displays the earth. So Marble can be used as a nice digital replacement for your desktop globe at home where you can look up places. But wait! There's more to it: Actually these days Marble can also display flat maps (thanks to Carlos Licea), can show different "map themes" and can serve as a Qt4-widget as well as an application! This means that as a programmer you can use Marble in your very own project as a map widget (License: LGPL). Marble was designed to run on any device and on any operating system supported by Qt4 without any further requirements. You can download the latest version of Marble together with KDE 4.0.1 here (It's part of the KDE-EDU module). How Marble stores texture data Read More

Akonadi and Nepomuk - Holding Hands in Osnabrück

Thursday, 7 February 2008
Last weekend I was invited to the KDE-PIM meeting in Osnabrück to represent Nepomuk. First of all I have to say: thanks a lot for inviting me, guys. The meeting was a lot of fun (although staying awake got harder during the course of the three days you crazy work-maniacs!) and it was great to see known faces again and meet new nice people. As they have during the last years Intevation hosted the event and I want to give a quick thanks to them, too. Read More

microgeek

Thursday, 7 February 2008
Lily Aimée Adam, 2955g, 51cm, healthy, impossibly lovely. Publicity shot here. Righteous.

Parloids

Thursday, 7 February 2008
Plasmoids have been creeping into different parts of KDE... The E-Team lately spotted two of them. So maybe check out KDE-Edu to get a real hot calculator made by apol. The gui is in need of some love, but it's already very powerful since it uses KAlgebra behind the scenes (yay, scientific calculations on your desktop). [image:3267] The other one I quickly hacked together, using artwork that leeo did for the icon originally. Being reminded by aseigo to seperate engine and applet, I had a glimpse at how engines are created, decided it's easy enough, so the engine is there too now. It can easily be extended to give tons of data, so I'm open to crazy ideas, if you want it to spit out more data, just tell me. Font config works, but some layouting might do good. And feel free to come up with an improved design and especially layout. As I have little time lately and rather want to get Parley in trunk into a working state again, I won't update the plasmoid much. Junior jobbing anyone? Great place to start, I already have a couple of ideas... drop by in #kde-edu on freenode. Read More