JUL
24
2018

2018 Toulouse PIM sprint report

Like every year, the PIM developers met in Toulouse for a bit of bugfixing ;)

Except for Dan Vratil, we forgot to blog about it, here's a belated blog post with what we did there.

Privacy goal
Memory hole support
The big issue here is that we need to update the headers outside the mimetreeparser, like subject or from/to etc. That's why mimetreeaparser should get an interface, so plugins can change headers. Another header that would benefit of that interface is a DKIM signature checks on the mail, that would update the DKIM status based on actual data.

KMail

  • Improved security when encountering encrypted emails with external references.
    D12391, D12393, D12394.
  • Discussed header validation support (memory hole, DKIM): how to integrate this into mimetreeparser, and how to feed that into the mail renderer. See this task.

Akonadi Search
Support for encrypted emails

kdepim dependencies
In order to move ksieve down the dependency tree:

  • pimcommon - get rid of PimCommon::SieveSyntaxHighlighterUtil (here and here)
  • PimCommon::Util::createImapSettingsInterface is also in AkonadiMime, so properly moved the function to PimCommonAkonadi at least, as it needs Akonadiserver to run anyways
  • for libSieve change dependency kdepim-runtime (imapresource <- libksieve) should make it easier to move libsieve to a lower tier and make imapresource push informations to ksiveui and we can remove akonadi dependency

KItinerary

  • try harder to extract boarding times from pkpass files
  • live test of the itinerary app passing manual and automatic boarding pass inspection on two airports
  • acquire more test data from sprint attendees (and as a result of this made EasyJet IATA BCBP data parsing work, and fixed Iberia boarding pass extraction)
  • handle merging of codeshare flight data
  • rework data extraction in KMail when multiple data elements are found (this is now all merged together to further improve the completeness of the data)
  • made boarding passes/tickets for trains and busses accessbile in the app too (this should in theory enable the app to let you travel on SNCF when booked via trainline.eu)
  • attach the entire reservation data to ical events created from it, so it's fully synced to other devices that way

Akonadi

  • finished and merged Notification Payloads (T639). The hope of Notification Payloads is to give us substantial performance boost and IO reduction by easing on the amount of SQL queries we do, especially during sync, by sending the changed entities as part of change notifications, so that clients don't have to query the entity from Akonadi after they receive the notification
  • improved Akonadi Console to simplify debugging notifications and monitors
  • new Logging tab in Akonadi Console with logs from all Akonadi apps
  • improve Debbugger view

Debugging akonadi
Update debugger window to handle more messages without slowing down the machine too much

LDAP completion
Now works again with non-ascii characters in the search string

WebEngine regressions
Investigated Qt 5.11 bug which broke clicking on links in KMail, reported to the Qt bugtracker, fixed since.

Qt performance
Sped up QTimeZone::isTimeZoneIdAvailable by a factor 43. This is relevant when deserializing many events from akonadi (testcase: opening an invitation email in standalone kmail, loads the full calendar)

IMAP Resource

  • Fix IMAP resource stuck if connection lost between OK and Login, as well as during Login
  • Fix possible data loss on connection lost, depending on the current task. This is likely what leads to some emails in a Send folder having no remoteId, making every future sync of that folder slower

kdepim addons
Fixed long-standing crash on make install, due to a memory leak when closing a separate viewer window.

KSMTP

  • Fix sending emails where a line starts with a dot
  • Remove dependency on KMime

Kontact
Prettier icons in the Kontact side pane

Turning PIM libraries into KDE Frameworks (ongoing effort)

  • Plan for syndication: remove FileRetriever (only used by akregator via Loader::loadFrom), that removes the KIO dependency, after 18.08 freeze ABI and move to KF5
  • kcontacts kdelibs4support data dependency: move to CLDR for country name mapping, move the address formatting data from kdelibs4support to kcontacts

OAuth login

  • moved the XOAUTH2 SASL plugin to LibKGAPI
  • IMAP: it's possible to use Gmail IMAP without the OAuth authentication again
  • added support for OAuth authentication to SMTP module
  • added XOAUTH2 support to KMailTransport, so it's possible to authenticate with Gmail SMTP via XOAUTH2 (which means 2FA and Gmail for Business domains support out of the box)

outreach
Accept and handle outstanding review requests from new contributors.

Ruqola (RocketChat client)

  • Fixing AboutPage tab bar
  • Fixing about page author layouting
  • Preparing fix for password security if prefilled
  • Discussion design over support for multiple accounts

Android backend for KCalCore

  • assumption: we use the platform calendaring API, rather than Akonadi, and use KCalCore::Calendar as the unifying API for that, backed by AkonadiCalendar and a yet to be created AndroidCalendar
  • challenge: that API doesn't cover all of ical, but for some sync connectors (namely DavDroid) some extra fields seem available in the ExtraProperties table (confirmed for GEO, X-xxx and attachments)
  • problem: OpenTasks + DavDroid do not store custom properties or parent <-> child relations, that's even lost on Android-side changes. This means we also can't access all information needed for Zanshin on Android.
  • what to do about the google calendar? any chance to access additional ical fields there?

Syndication
Removed KIO and KCoreAddons dependency to make it ready to be moved to Frameworks

Modern C++ usage
KF5 is still on C++11, due to GCC 4.8 and MSVC 2013
Application code could use C++14 (e.g. generic lambdas), which would require GCC 4.9 and MSVC 2015, which is fine

Akregator

  • cleanups on the storage plugin API
  • drop deprecated unused code
  • work on a SQL backend (private branch so far, but beginning to work)

Community Data Analysis
See Kévin's blog post.

Many thanks to all our sponsors and donators for making such sprints possible.

Comments

Hi, do you have any plans to support accent free search in messages from KMail? thanks, m.


By Michal Hlavac at Tue, 07/24/2018 - 08:23

That would be a useful feature, but I'm not sure how to implement it :-)


By David Faure at Tue, 07/24/2018 - 09:13

Thanks for the update.

Did you, by all means have a look at a kmail regression bug 395711 during the sprint?
It prevents kmail to load external images.
The link to the bug report is: https://bugs.kde.org/show_bug.cgi?id=395711
Would be convenient when this regression is fixed.


By Richard at Tue, 07/24/2018 - 10:41

This would have required a time machine, since the sprint was in April and the bug report is from June ;)
My sprint report is very late, sorry about the confusion.

But thanks for the reference to that bug report, I'm seeing that regression too, good to know it's been reported.


By David Faure at Tue, 07/24/2018 - 11:15

Did you discuss the issue of support for kmail? It's a very important application, and users should always be able to rely on it.

When reading for example this bug report https://bugs.kde.org/show_bug.cgi?id=387931#c16 the reliability (from version to version, or upgrade after upgrade) could be questioned.

From my own recent experience; I just upgraded to openSUSE Leap 15.0 and kmail in this release seems to function less well, compared to the one I came from (openSUSE 42.3). It's for this reason that I started to look for similar bugs.

Till now I did not discover new functionality, hopefully the code has been cleaned under hood, which is a good and necessary improvement, but one that is not visible for the end user of course.

Would I use kmail in a company (I don't) I would be hesitant to upgrade. This should not be case in such an environment...

Looking forward to a stellar PIM package!


By Richard at Tue, 07/24/2018 - 13:12

I can't read anything regarding Kube/Sink here. Is that effort only worked on by Cristian Mollekopf alone?
Is Kube/Sink the future or those just alternatives to kmail / KDE PIM?


By Esokrates at Wed, 07/25/2018 - 09:30

Correct, Kube and Sink are not efforts from the KDE community at large.

I personally don't believe they are the future in any way.


By David Faure at Wed, 07/25/2018 - 09:34