Akonadi misconception #1: where is my data?
I regularly see the same misconception and fear popping up on the mailing lists, bug reports and IRC: if the Akonadi database gets corrupted, I will lose my data.
To make it clear from the beginning: the Akonadi database is NOT your MAIN data storage.
So what is the database?
1) It is an interface: the Akonadi server and the underlying database is a common interface to your (PIM-alike) data for different applications, so those applications do not have to deal with the data files directly.
2) But I see my email headers and even the body in the database and in $HOME/.local/share/akonadi/file_db_data. Why? Because the database is also a cache towards your data. Common, frequently accessed parts (like e-mail headers) are stored in the database. These are usually stored permanently and kept in sync with your original data source (IMAP folders, mails on the local disc).
3) Is there anything I might lose by deleting the database? Yes, there is, and that is the metadata added to your data. That can be anything extra information that cannot be stored in the format of your data, like Nepomuk tags or other custom information. In case of emails, you might think that read/forwarded/etc. can be lost. Luckily this is not the case (since KDE 4.7.2), as the email storage formats can store these informations natively.
The above explains why you will not lose any critical information by losing your akonadi database.
Good, but where is my data if not in the database? This depends on what kind of data we are talking about.
1) E-mail: in case of IMAP (online or disconnected) your data is on the IMAP server. With disconnected IMAP there are windows when your local cache contains data that is not yet syncronized to the server, deleting the local cache in this case indeed will make you lose the unsynchronized files. This is not Akonadi specific though, this is true for any disconnected IMAP client.
2) Calendars and contact information: they can be either on a server (Kolab server, LDAP server) and only cached in Akonadi as explained, or they can be in local vcard or .ics file. The actual location of these files again depends on your setup. The new standard place for them is $HOME/.local/share/contacts.
Still there were reports of data losing, why? Unfortunately programmers are not perfect and introduce bugs in the codebase. One of the most severe bugs caused real data losing when copying mails from one folder to another. This is fixed with KDE 4.7.2+ and any recent Akonadi server. There are other bugs around, but none will cause losing your original data files.
Finally, what will happen if the database gets corrupted? Of course, it needs to be recreated. You can try by shutting down akonadi server (akonadictl stop), removing the $HOME/.local/share/akonadi and syncronize all your resources again (this will take some time). If the database is not recreated, you need to do a full cleanup by removing also the configuration files under $HOME/.config/akonadi.
I hope this will clear some confusion about the data storage inside Akonadi.
And a few word about the database itself.
What about other database backends? There were plans to use Virtuoso, to reduce the number of database severs needed for a KDE desktop, but the work was not completed. See the techbase aritcle.
UPDATE: Christophe Giboudeaux raised a point about PostgreSQL, that their database format changes between major releases in an incompatible way and there is no automated, easy way for upgrade (you need to dump it with the old version and import with the new one). Sad, that there is no perfect solution.
I for one would love to see
I for one would love to see virtuoso used as a backend. If more apps manage to move to stringi/nepomuk for indexing then we could at least theoretically have just one database for "everything".
Postgres has an in-place upgrade tool since a couple of versions ago. It is not automatic (thankfully: it's offline, one-way, potentially ressource-intensive, probably not what you want yet, etc) but could be automated by akonadi itself (since akonadi knows about the usage pattern, it can make those upgrade decisions on behalf of the user). Taking a db dump (as part of the regular backup process) wouldn't hurt anyway.
I'll try akonadi with postgres again. I had tried a long time ago, but the many pim bugs at the time prompted me to stick with the recomended option. I trust pg much more than mysql and have it runing on my machines for many projects, whereas mysql is only there for akonadi. So switching to postgres would improve my resource usage, maintenance burden, and peace of mind. YMMV.
But where are my emails?
Thanks for the explanation.
Regarding the email storage, I am still puzzled, though... I would also expect to find my emails somewhere on the local drive but I can only find them in the database.
I fetch my emails using POP and already try to search in the usual locations ($HOME/Mail, $HOME/.local/share/.local-mail, etc...). Plus there are a few Gigabyes so it should not be that easy for them to hide :)
Do you have an idea? Seems to be a configuration issue but I have not found any solution so far.
Find the folder you are looking for in KMail. The toplevel entry of the folder tree should be something like Local Folders (unless you renamed it). Find the Local Folders account in the KMail account configuration. Click on Modify and see where it points to.
In case you have multiple Local Folders or similar accounts, look at the POP3 accounts configuration, see in the Advanced tab where does it deliver the mail and find the account that corressponds to that folder.
Local Folders and KMail Folders
Thanks a lot for your answer!
My Emails are in "KMail Folders" (which exists besides "Local Folders") and I cannot access to its configuration (I did the migration several months ago).
I can maybe try to copy all my emails from "KMail Folders" into "Local Folders" and try somehow to remove "KMail Folders".
Do you have any idea?
Strange you couldn't access
Strange you couldn't access the configuration. Double click on Modify should get you there. KMail Folders means it is a migrated account, so the mails should be just where your mails were prior to migration (migration doesn't copy the mails themselves). Of course you can remove the account and add a new one back, but for that you'd need to know where the mails are. Try a global search in your $HOME (including the hidden subdirs).
You can also post on [email protected], probably easier to discuss there then through a blog.
Akonadi still broken? Kubuntu 15-10
I installed the recent kubuntu version on two machines.
One instance is still working (e.g. Kmail works as it should),
the other one broke down after system upgrade (Akonadi not starting).
Possibly the mySQL bug, but not usable. I was longing to remigrate back from thunderbird to kmail,
but akonadi just kills me.
Apart from upgrade, these might be changes that might have caused my problem:
1. soft link from /home to /data/home
2. changed root password.
KMail is showing duplicates
KMail is showing 407 emails in one folder, when there are only 199 emails for that folder on disk. When I go to the Akonadi console, I can see duplicate references to the one (Kmail) file. How do I 'force' Akonadi to reload from disk, or sync with KMail for the one folder ?
How to backup?
I am currently setting up a tape backup. Considering that there are lots of files in ~/.local/share/akonadi/db_data/ and more importantly - that it is a live database - what is the proper way to back that up? Is it safe to simply copy all the files? Or how to do a db dump. MySQL's 'show databases' doesn't show any akonadi DB.
upgrading mysql for akonadi
I am getting a lot if errors in the log. mariadb 10.xx.xx has a different structure than the version before. The problem is to solve it by running upgrade. I did by doing mysql_upgrade -p
but that doesn't upgrade mysql akonadi is using. How should that be done?