2018 Toulouse PIM sprint report

Tuesday, 24 July 2018  |  David Faure

#PIM  

Like every year, the PIM developers met in Toulouse for a bit of bugfixing ;)

Except for Dan Vratil, we forgot to blog about it, here's a belated blog post with what we did there.

Privacy goal Memory hole support The big issue here is that we need to update the headers outside the mimetreeparser, like subject or from/to etc. That's why mimetreeaparser should get an interface, so plugins can change headers. Another header that would benefit of that interface is a DKIM signature checks on the mail, that would update the DKIM status based on actual data.

KMail

  • Improved security when encountering encrypted emails with external references. D12391, D12393, D12394.
  • Discussed header validation support (memory hole, DKIM): how to integrate this into mimetreeparser, and how to feed that into the mail renderer. See this task.

Akonadi Search Support for encrypted emails

kdepim dependencies In order to move ksieve down the dependency tree:

  • pimcommon - get rid of PimCommon::SieveSyntaxHighlighterUtil (here and here)
  • PimCommon::Util::createImapSettingsInterface is also in AkonadiMime, so properly moved the function to PimCommonAkonadi at least, as it needs Akonadiserver to run anyways
  • for libSieve change dependency kdepim-runtime (imapresource <- libksieve) should make it easier to move libsieve to a lower tier and make imapresource push informations to ksiveui and we can remove akonadi dependency

KItinerary

  • try harder to extract boarding times from pkpass files
  • live test of the itinerary app passing manual and automatic boarding pass inspection on two airports
  • acquire more test data from sprint attendees (and as a result of this made EasyJet IATA BCBP data parsing work, and fixed Iberia boarding pass extraction)
  • handle merging of codeshare flight data
  • rework data extraction in KMail when multiple data elements are found (this is now all merged together to further improve the completeness of the data)
  • made boarding passes/tickets for trains and busses accessbile in the app too (this should in theory enable the app to let you travel on SNCF when booked via trainline.eu)
  • attach the entire reservation data to ical events created from it, so it's fully synced to other devices that way

Akonadi

  • finished and merged Notification Payloads (T639). The hope of Notification Payloads is to give us substantial performance boost and IO reduction by easing on the amount of SQL queries we do, especially during sync, by sending the changed entities as part of change notifications, so that clients don't have to query the entity from Akonadi after they receive the notification
  • improved Akonadi Console to simplify debugging notifications and monitors
  • new Logging tab in Akonadi Console with logs from all Akonadi apps
  • improve Debbugger view

Debugging akonadi Update debugger window to handle more messages without slowing down the machine too much

LDAP completion Now works again with non-ascii characters in the search string

WebEngine regressions Investigated Qt 5.11 bug which broke clicking on links in KMail, reported to the Qt bugtracker, fixed since.

Qt performance Sped up QTimeZone::isTimeZoneIdAvailable by a factor 43. This is relevant when deserializing many events from akonadi (testcase: opening an invitation email in standalone kmail, loads the full calendar)

IMAP Resource

  • Fix IMAP resource stuck if connection lost between OK and Login, as well as during Login
  • Fix possible data loss on connection lost, depending on the current task. This is likely what leads to some emails in a Send folder having no remoteId, making every future sync of that folder slower

kdepim addons Fixed long-standing crash on make install, due to a memory leak when closing a separate viewer window.

KSMTP

  • Fix sending emails where a line starts with a dot
  • Remove dependency on KMime

Kontact Prettier icons in the Kontact side pane

Turning PIM libraries into KDE Frameworks (ongoing effort)

  • Plan for syndication: remove FileRetriever (only used by akregator via Loader::loadFrom), that removes the KIO dependency, after 18.08 freeze ABI and move to KF5
  • kcontacts kdelibs4support data dependency: move to CLDR for country name mapping, move the address formatting data from kdelibs4support to kcontacts

OAuth login

  • moved the XOAUTH2 SASL plugin to LibKGAPI
  • IMAP: it's possible to use Gmail IMAP without the OAuth authentication again
  • added support for OAuth authentication to SMTP module
  • added XOAUTH2 support to KMailTransport, so it's possible to authenticate with Gmail SMTP via XOAUTH2 (which means 2FA and Gmail for Business domains support out of the box)

outreach Accept and handle outstanding review requests from new contributors.

Ruqola (RocketChat client)

  • Fixing AboutPage tab bar
  • Fixing about page author layouting
  • Preparing fix for password security if prefilled
  • Discussion design over support for multiple accounts

Android backend for KCalCore

  • assumption: we use the platform calendaring API, rather than Akonadi, and use KCalCore::Calendar as the unifying API for that, backed by AkonadiCalendar and a yet to be created AndroidCalendar
  • challenge: that API doesn't cover all of ical, but for some sync connectors (namely DavDroid) some extra fields seem available in the ExtraProperties table (confirmed for GEO, X-xxx and attachments)
  • problem: OpenTasks + DavDroid do not store custom properties or parent <-> child relations, that's even lost on Android-side changes. This means we also can't access all information needed for Zanshin on Android.
  • what to do about the google calendar? any chance to access additional ical fields there?

Syndication Removed KIO and KCoreAddons dependency to make it ready to be moved to Frameworks

Modern C++ usage KF5 is still on C++11, due to GCC 4.8 and MSVC 2013 Application code could use C++14 (e.g. generic lambdas), which would require GCC 4.9 and MSVC 2015, which is fine

Akregator

  • cleanups on the storage plugin API
  • drop deprecated unused code
  • work on a SQL backend (private branch so far, but beginning to work)

Community Data Analysis See Kévin's blog post.

Many thanks to all our sponsors and donators for making such sprints possible.