Querying a Wiki Using Qt

    rich's picture
    2009
    11
    Jan

    I spent a little time yesterday working on a class that provides
    a wrapper around the MediaWiki API. It makes quite a nice example
    of how the QNetworkAccessManager API can be combined with
    QXmlStreamReader to quickly access web services. The code, as you'll
    see, is very simple.

    First lets take a look at the constructor and the data members. We
    specify the path to the api.php file of the wiki we want to query,
    and setup the network access manager.


    struct MediaWikiPrivate {
    QStringList results;
    QUrl apiUrl;
    QNetworkAccessManager *manager;
    int maxItems;
    };

    MediaWiki::MediaWiki( QObject *parent )
    : QObject( parent )
    {
    d = new MediaWikiPrivate;
    d->apiUrl = QUrl("http://en.wikipedia.org/w/api.php");
    d->manager = new QNetworkAccessManager( this );
    d->maxItems = 10;
    connect( d->manager, SIGNAL(finished(QNetworkReply*)), SLOT(finished(QNetworkReply *)) );
    }

    People using the class can ask us to perform a search by calling
    the search() method with the term they want to look for. Again,
    the code is simple.


    void MediaWiki::search( const QString &searchTerm )
    {
    QUrl url = d->apiUrl;
    url.addQueryItem( QString("action"), QString("query") );
    url.addQueryItem( QString("format"), QString("xml") );
    url.addQueryItem( QString("list"), QString("search") );
    url.addQueryItem( QString("srsearch"), searchTerm );
    url.addQueryItem( QString("srlimit"), QString::number(d->maxItems) );

    qDebug() << "Constructed search URL" << url;

    d->manager->get( QNetworkRequest(url) );
    }

    We use the addQueryItem() method of QUrl to construct our search, by
    specifying each of the options we need. Finally, we launch the request
    by calling the get() method of QNetworkAccessManager.

    Once the network request is completed, our finished() slot is called.
    We check the status to see if the request succeeded then pass the
    reply to the processSearchResult() method. Note that QNetworkReply
    is a subclass of QIODevice which makes it very easy for us to work
    with.


    void MediaWiki::finished( QNetworkReply *reply )
    {
    if ( reply->error() != QNetworkReply::NoError ) {
    qDebug() << "Request failed, " << reply->errorString();
    emit finished(false);
    return;
    }

    qDebug() << "Request succeeded";
    bool ok = processSearchResult( reply );
    emit finished( ok );
    }

    bool MediaWiki::processSearchResult( QIODevice *source )
    {
    d->results.clear();

    QXmlStreamReader reader( source );
    while ( !reader.atEnd() ) {
    QXmlStreamReader::TokenType tokenType = reader.readNext();
    // qDebug() << "Token" << int(tokenType);
    if ( tokenType == QXmlStreamReader::StartElement ) {
    if ( reader.name() == QString("p") ) {
    QXmlStreamAttributes attrs = reader.attributes();
    //qDebug() << "Found page" << attrs.value( QString("title") );
    d->results << attrs.value( QString("title") ).toString();
    }
    }
    else if ( tokenType == QXmlStreamReader::Invalid )
    return false;
    }

    qDebug() << "Results" << d->results;
    return true;
    }

    To parse the XML returned by the wiki, we use QXmlStreamReader. We're only
    interested in elements that look like this:

       <p ns="0" title="Qt (toolkit)" />
    

    So we locate those when we see a StartElement token, then extract the title
    attribute. Once this process is complete, we have a list of all the pages
    matching the query and emit our finished() signal.

    To test the code I've written a small wrapper that lets you enter a term,
    then searches online when you press return. The code queries wikipedia right
    now, but simply by changing the setApiUrl() call you can tell it to search
    techbase. I've left that code in place but commented out so people can try it.

    In future, this code could be used as a KRunner plugin, as part of an IDE,
    or more generally anywhere else where someone wants to query a wiki in their
    application. The full example can be downloaded from
    http://xmelegance.org/devel/mediawiki/wikiviewer.tar.bz2. Enjoy.

    Comments

    Comment viewing options

    Select your preferred way to display the comments and click "Save settings" to activate your changes.
    tstaerk's picture

    great

    great - this will help me to make my program "mediawiki2mindmap" perfect!

    You can really give commands to mediawiki like this: http://en.wikipedia.org/w/api.php?action=sitematrix

    cheers

    Thorsten

    daniel molkentin's picture

    API docs

    For those who are curious: The API documentation for Mediawiki is available at http://www.mediawiki.org/wiki/API.

    Comment viewing options

    Select your preferred way to display the comments and click "Save settings" to activate your changes.