JAN
11
2009

Querying a Wiki Using Qt

I spent a little time yesterday working on a class that provides
a wrapper around the MediaWiki API. It makes quite a nice example
of how the QNetworkAccessManager API can be combined with
QXmlStreamReader to quickly access web services. The code, as you'll
see, is very simple.

First lets take a look at the constructor and the data members. We
specify the path to the api.php file of the wiki we want to query,
and setup the network access manager.


struct MediaWikiPrivate {
QStringList results;
QUrl apiUrl;
QNetworkAccessManager *manager;
int maxItems;
};

MediaWiki::MediaWiki( QObject *parent )
: QObject( parent )
{
d = new MediaWikiPrivate;
d->apiUrl = QUrl("http://en.wikipedia.org/w/api.php");
d->manager = new QNetworkAccessManager( this );
d->maxItems = 10;
connect( d->manager, SIGNAL(finished(QNetworkReply*)), SLOT(finished(QNetworkReply *)) );
}

People using the class can ask us to perform a search by calling
the search() method with the term they want to look for. Again,
the code is simple.


void MediaWiki::search( const QString &searchTerm )
{
QUrl url = d->apiUrl;
url.addQueryItem( QString("action"), QString("query") );
url.addQueryItem( QString("format"), QString("xml") );
url.addQueryItem( QString("list"), QString("search") );
url.addQueryItem( QString("srsearch"), searchTerm );
url.addQueryItem( QString("srlimit"), QString::number(d->maxItems) );

qDebug() << "Constructed search URL" << url;

d->manager->get( QNetworkRequest(url) );
}

We use the addQueryItem() method of QUrl to construct our search, by
specifying each of the options we need. Finally, we launch the request
by calling the get() method of QNetworkAccessManager.

Once the network request is completed, our finished() slot is called.
We check the status to see if the request succeeded then pass the
reply to the processSearchResult() method. Note that QNetworkReply
is a subclass of QIODevice which makes it very easy for us to work
with.


void MediaWiki::finished( QNetworkReply *reply )
{
if ( reply->error() != QNetworkReply::NoError ) {
qDebug() << "Request failed, " << reply->errorString();
emit finished(false);
return;
}

qDebug() << "Request succeeded";
bool ok = processSearchResult( reply );
emit finished( ok );
}

bool MediaWiki::processSearchResult( QIODevice *source )
{
d->results.clear();

QXmlStreamReader reader( source );
while ( !reader.atEnd() ) {
QXmlStreamReader::TokenType tokenType = reader.readNext();
// qDebug() << "Token" << int(tokenType);
if ( tokenType == QXmlStreamReader::StartElement ) {
if ( reader.name() == QString("p") ) {
QXmlStreamAttributes attrs = reader.attributes();
//qDebug() << "Found page" << attrs.value( QString("title") );
d->results << attrs.value( QString("title") ).toString();
}
}
else if ( tokenType == QXmlStreamReader::Invalid )
return false;
}

qDebug() << "Results" << d->results;
return true;
}

To parse the XML returned by the wiki, we use QXmlStreamReader. We're only
interested in elements that look like this:

   <p ns="0" title="Qt (toolkit)" />

So we locate those when we see a StartElement token, then extract the title
attribute. Once this process is complete, we have a list of all the pages
matching the query and emit our finished() signal.

To test the code I've written a small wrapper that lets you enter a term,
then searches online when you press return. The code queries wikipedia right
now, but simply by changing the setApiUrl() call you can tell it to search
techbase. I've left that code in place but commented out so people can try it.

In future, this code could be used as a KRunner plugin, as part of an IDE,
or more generally anywhere else where someone wants to query a wiki in their
application. The full example can be downloaded from
http://xmelegance.org/devel/mediawiki/wikiviewer.tar.bz2. Enjoy.

Comments

For those who are curious: The API documentation for Mediawiki is available at http://www.mediawiki.org/wiki/API.


By Daniel Molkentin at Sun, 01/11/2009 - 15:31

great - this will help me to make my program "mediawiki2mindmap" perfect!

You can really give commands to mediawiki like this: http://en.wikipedia.org/w/api.php?action=sitematrix

cheers

Thorsten


By tstaerk at Mon, 01/12/2009 - 17:47