For the past few days I was (besides watching GUADEC talks) experimenting a little with Zeitgeist and natural language processing... or sort of anyway. Having no real prior knowledge in NL field and not using any of the existing NLP libraries (as I couldn't find anything in C), definitely made it interesting, but also made me realize that NLP is really hard (even though I only wanted to get a very specific app to work) and taking this path most likely isn't a way to get somewhere.

But anyway, the original idea was to make an algorithm which would take a natural language query and "compile" a Zeitgeist event template from it. This would make it possible to basically ask questions about stuff you did on your computer (not necessarily in a question-form) and get results back from Zeitgeist. The way I did the algorithm was very easily pluggable into any ZG application, so of course I tried it with our lovely Sezen search applet, and on the following screenshots you can see it in action:


If you for some reason don't see the images, there are queries like "music played today", "web pages accessed on wednesday" and "files modified 1 week ago". Those are basically types of strings that my simple engine is able to process right now (besides simple queries like "movies", "vector images", etc.)

I won't deny that all this work got inspired by seeing screenshots for the "Storage" project which Siegfried dug out from somewhere. And even though it seems to be long abandoned and dead, I'd still love to see its sources, but unfortunately I couldn't find them anywhere... But if you know about an URL where it still lives, please give me a shout. ;)

Also, if nothing else, this work led to a patch for Vala which fixes up bindings for N-ary trees, so soon one will be able to finally use the N-ary trees' datatypes present in glib, without having to reimplement it in Vala.

There has been quite some work on both Sezen and the panel applet since my last post, and here's what's new on that front:

We went through quite a few design iterations of the applet, here is the evolution:
At first, we had the main Sezen widget which was stuffed in a menu, but as I mentioned in my previous post, I found this sub-optimal, as navigating in menu is not the same as doing so in a standalone window, and therefore I wanted the browser-like menu, which came to life in the third iteration, but the problem with it was that even though we had thumbnails (though not in the image), they were totally unhelpful, as they were too small. So the fourth iteration introduced three rows of text per each item, which allowed a big thumbnail on the right, but then some items had only one icon on the left, some had one on left and one on the right, and this just felt weird, so I tried to remove the icons from the left and have all of them on the right, but as you can see in the fifth image, this also doesn't look right, so the idea of thumbnails on the right was abandonded, and we ended up with medium-sized thumbnails on the left as you can see in the last image.

There are still a few loose ends in the applet (clicking on the scroll bar doesn't work most of the time, since scrollbar was never meant to be inside menu, and therefore isn't trivial to fix), and it could also use a global hotkey to popup the search. But even now I find it very usable if one has enough stuff logged by Zeitgeist.

Since Seif still thought that standard Sezen window is the way to go (with which I obviously don't agree), there's also another version of the applet, which just opens undecorated Sezen under the menu item position:
And yea, I agree that it looks nice (especially with elementary theme), but is practically unusable only with keyboard, which I find a deal breaker (also it doesn't close if you click some other window... though this could be implemented, standard menu does it automatically).

Now I wonder about the future of the applet, should we try to fix the issues it has and push it upstream, or turn it into a widget which any app can use (it'd just tell us which mimetypes it's interested in and it would augment the "Open file" function). Also which version is really better? (but please judge by using them, not by looking at the screenshots)

Links to get the code:
https://code.launchpad.net/~elementaryart/gnome-applets/sezen
https://code.launchpad.net/~elementaryart/gnome-applets/sezen2

To install you need to get the branch, its dependencies and run:

As the reports themselves are quite boring for many people, I'll try to post on the blog more interesting stuff, so it will no longer be a direct copy of the message sent to gnome-soc-list. So let's give it a try...

People who read my reports know that lately I've been working on Sezen (if you haven't heard about this awesome piece of software using Zeitgeist go see a couple of videos at Seif's blog), where both Seif's python version and mine Vala version is receiving lots of love - Seif started to use Mikkel's FTS extension for the search, I didn't do this yet (as the FTS extension might not be installed), but I improved the search we had and now it's no longer that stupid (ie doesn't treat everything as exact phrase search, instead supports "quoted phrase search").


But as you could notice from the title of this post, we went further and tried to integrate sezen into gnome-panel via an applet, and this is how far I got:

At first Seif wanted to show the full sezen window also in the panel applet, but I wanted to try more "panely" look and feel and therefore went this way. Not to mention the challenge it was to get the Entry and the scrollable IconView widgets to work inside a menu (thank you Gtk for this unforgettable experience, you won a few battles, but I gotcha anyway).
If you're thinking now that it doesn't look as blingy as standalone Sezen I have to agree, but I still think that it has more integrated look with the rest of the panel and is already quite usable. I've been thinking where we could take this and I keep looking at this:

Wouldn't it be cool to have this kind of widget on your panel allowing you to start working with anything ZG knows about? What do you think?

For those who are interested, you can also read my official GSoC report #6.

Hey!

As planned, this week I finished our new Chrome extension together with the NPAPI plugin. In the end the javascript part of the extension didn't turn out to be as straightforward as I expected, but using some not-so-nice hacks  it does what it's supposed to do. After I finished the Chrome extension I tried to make our new totem plugin using libzeitgeist build out of totem's tree, and fortunately this was easier than I thought it would, so it's done now and I also got some time to start to revamp the build system in zeitgeist-dataproviders [1], and even though now it's a strange mix of autotools in the top source directory and our own Makefiles in the plugin dirs, it works and currently provides the ability to detect which plugins can be build and it builds only those (note that it's incomplete, but the framework is there).

Plan for the next week is to move the whole build system to autotools, with the ultimate goal of making it possible to build packages of the dataproviders. Once that is done, I'll start to write another totem plugin, this time one which will pull data from ZG and therefore will add some UI elements to totem. Stay tuned ;)

Note to self: maybe I'm too spoiled by python, but this just doesn't work in JS (at least not when writing Chrome extension), even though there's no warning/error:

if (condition) {
    function callback () {
        console.log("You'll never see this");
    }
    document.addEventListener("event", callback, false);
}

[1] https://code.launchpad.net/zeitgeist-dataproviders

Hey everyone!

This week I've been busy with traveling and moving, but during the past few days I finally managed to work, so here's what I did:

I started implementing Zeitgeist dataprovider for Chrome (using NPAPI), but this turned out to be quite hard, as after the initial implementation the plugin was working fine in Firefox, but didn't want to show up in Chrome, there weren't any error messages and strace didn't even show Chrome trying to open the dynamic library (and strace -f was hanging)... So it was quite "fun"! Anyway today I managed to make it work and now we have a first version of Chrome extension!


Plan for next week is:

• finish the Chrome extension, so it doesn't send multiple events for one website visit as it does now... Also does anyone know how to get document mime-type in Chrome?
• if time allows, I'll try to make our new Totem extension to build out of totem tree, which might not be that easy to do (actually I'm not sure it's even possible)

Zeitgeist can already make users lives really easy (see my last Awn + Zeitgeist post), but we don't have to stop there, one of things I'd really like to see would be if applications helped us with this - if the apps expose which URI they are currently working with (which file is currently open / web page currently viewed / ...) we can make very precise queries as to which other files (or contacts, web pages, applications etc.) are related to this URI.

As everyone likes examples imagine that you're working with a spreadsheet and zeitgeist knows that the last time you were working with it you were checking www.sitewithstats.com, had last_years_report.pdf open and were talking to xyz. Therefore it will provide you these options in some kind of menu (or dock / indicator / windicator). Wouldn't it be great?

To make this possible I suggest new X property for windows (let's say it'd be "_NET_WM_CURRENT_URI") and the only thing required by the apps is to set this property when a file is opened and update it when appropriate. If people like this idea I can prepare a patch for gdk (adding gdk_window_set_current_uri_hint) and libwnck (which would expose the property for pagers). Or perhaps there is already something similar what I missed?

Comments?

People say that image is worth thousand words, so take a look at this video instead of me saying all those words:



Under the hood there's Awn's window to desktop file matching backed by wncksync (if available). And of course Zeitgeist with a couple of extensions which I'm currently working on.

Before anyone asks, this isn't yet pushed anywhere is now available in a separate awn-extras branch (https://code.launchpad.net/~mhr3/awn-extras/zeitgeist-applet), and will be merged to awn-extras trunk once there's an official release of libzeitgeist and everyone (read awn-testing PPA users) will be able to play with it.

I was looking at interesting projects where I could participate in Google Summer of Code, and the project that caught my attention was libzeitgeist - a C-based library (with Vala bindings!) wrapping the quite complex DBus interface of Zeitgeist, so I wrote a proposal and even though there were some complications and modifications, it was accepted!
Therefore this summer I'll be spending lots of time making sure that the apps that you regularly use in GNOME will have plugins which talk to Zeitgeist, so it can gather as much context of your activities as possible.

Thanks to Mikkel and Seif for loads of help already, but be aware guys that it was just the beginning :D

Quick introduction for new Planet GNOME readers: I’m a high school student in Ra’anana, Israel and have been involved in GNOME ever since GHOP 2007. After attending the GNOME UI hackfest this past October, I cofounded Zeitgeist and have been working on various new user interfaces since then. Those of you who attended the GNOME Boston Summit may also remember me from my lightning talk on problems with developer documentation.

A few weeks ago I blogged about the following mockup:

I’m delighted to announce that it’s no longer just an idea! I’ve already started coding an early prototype called “Feedgeist” and the code will be on launchpad by the end of the week.

We’re planning on building sharing and collaboration features into Feedgeist. I’ll post a few more mockups on that soon.

I’ve been thinking about the proper way to handle fulltext searches in Zeitgeist. (E.g. searching through a file’s content instead of just searching for files by name.)

I think the best solution is to fall back to Tracker and other search engines for all text searches. This is especially important if we index things like GMail emails where we can’t possibly handle all searches ourselves without downloading and indexing all emails. (Instead, we would just index each email’s title and sender. If the user performs a search, we would use GMail’s API to find results on the fly.)

On that note, I’m planning on adding Google Documents and Flickr support. This probably wont happen until later this week or the beginning of next week