MehtaHurd

Recently, I saw a presentation on DeepaMehta. Sounds quite promising: Completely new approach instead of the traditional (broken) desktop metaphor; adapted to the way we think... Of course, I was mighty sceptical. It turned out more interesting than I expected.

Putting it bluntly, DeepaMehta is kind of an extensible filesystem browser with an advanced bookmarking/navigation system.

Of course, this statement is a misrepresentation in several regards. Most notably, DeepaMehta's objects aren't really meant as simple bookmarks. They can store considerable amounts of metadata, as well as actual data, and only optionally can reference traditional files or other external data, but normally they exist on their own -- in fact, the idea is that ideally (almost?) all data is stored in DeepaMehta itself. Also, DeepaMehta doesn't care at all about the traditional filesystem structure... Well, let's get more specific.

DeepaMehta is based on three core observations regarding existing desktop systems. One is that overlapping windows are an abomination that needs to be avoided. This is a very important issue of course. DeepaMehta always displays only one navigation window, and one window presenting the content of the currently active object -- just like Norton Commander in quick view mode... (Somewhat resembling Oberon, which also has a content pane plus another pane for other stuff -- though the division is different there.)

Limiting to always just one content window is a bit extreme IMHO: Seeing several things at the same time can be useful sometimes, given a sane (tiled and dynamic) window manager. Nevertheless, it's a Good Thing (TM) that DeepaMehta addresses the problem of overlapping windows.

The second observation is that traditional applications make no sense. Instead, we need a generic shell application, with plugins to be able to perform specific actions on various object types -- again a very important issue. To some extent, we see this happening with things like Nautilus already; but DeepaMehta is much more consequential: All navigation actions are performed in the navigation window, using standard mechanisms supported by plugins for creating specific object associations; and all object viewing/editing actions are performed in the content window, again supported by plugins for specific object types.

The third observation is that traditional navigation mechanisms are quite unsuitable to the way we work; for many tasks, a new approach can be vastly more efficient and convenient. This is the strong side of DeepaMehta, and the really innovative part: It uses a navigation system based on mind maps. Basically, you have all your objects, as well as various types of bilateral associations between them.

As there are far too many objects and associotions to make this manageable directly, you never look at all of them at once, but instead use various partial views (maps). A new view can be created by starting from some object, uncovering the associations (and associated objects) you are presently interested in, and going on from there, until you see all you want in this view. Once you have a view with all the objects interesting for your present occupation, you can easily swith which of these objects is visible in the content window at any given time, by focusing it in the map.

Constructing a map can also happen automatically, e.g. when browsing a web site in the content window: Each time you click a link, the linked site and the link association is added to the map. Othewise, creating a map by hand can quite tedious. (Especially as DeepaMehta, while introducing totally different high-level concepts, is very old-fashioned at the level of actual UI elements -- all the map management for example is done through nested context menues, requirering lots of searching and endless clicking...)

For some tasks, this seems way too static: To efficiently browse a filesystem (or generally to quickly move through any larger object structure), some more automated approach seems necessary. Not sure how to do that; but I think it should be possible to come up with something nicely extending the concepts in this direction.

There are some more fundamental problems with the existing DeepaMehta implementation, however. (Aside from the fact that it is written in Java...)

For one, it is centered around the idea that the future is in the net, with all intelligence in servers, and workstation machines being only dumb clients -- a pretty absurd notion. In spite of all the babbling of dotcom freaks, it's pretty obvious in any realistic view that the local system should always be the primary focus of a desktop framework. Networking is a mostly orthogonal issue, which can be nicely integrated using other mechanisms, for example a transparently networked filesystem like Plan9. Designing a desktop system as client-server from the ground up, only makes it more complicated and less flexible, creating much more problems than it solves.

The second major problem with DeepaMehta is that it creates a world of its own, with little relation to existing environments. This obviously makes transition pretty hard. Moreover, as it's impossible (or at least very unrealistic) to run a whole operating system, including administration etc., with DeepaMehta alone, there will always be the DeepaMehta layer on top/beside the rest of the system; and like with any sub-plattform, this creates very serious integration problems.

For these reasons, I don't consider DeepaMehta as is very useful in a broader view. Yet, the navigation approach seems immensly valuable. So, how could it be integrated at a generic level in more traditional systems? I have a number of ideas for various methods how to implement the DeepaMehta navigation concept through extensions to the normal filesystem. (The Hurd architecture makes such extensions easy...)

The basic idea is, instead of having a special object database like DeepaMehta, to use traditional filesystem entities (files and/or directories) as the primary objects. We already have links in the filesystem, so one might imagine something based on that for the object associations.

In a very crude approach, the DeepaMehta semantics could be expressed by some defined structure in a traditional filesystem. Every object would need to have its own directory, containing the main file as well as the object type designator, additional attributes, and object associations. An association could be represented by a link referencing the associated object, plus some file or a second link denoting the object type. (The object type itself can be described by a special object...) Maps could be either stored in additional subdirs inside the object directories, with pointers (links) to the associations visible in the particular map; or in completely distinct directories, linking all the objects and associations visible in the map.

This approach should work fine with programs that are aware of the special semantics. However, it creates lots of other problems otherwise. If you access the objects with normal programms, you always have to specify the main file inside the object dir. If you copy or delete objects, you have to do it recursively on the object dir. Generally, it's very complicated and not really intuitive. Also, it requires a special structure, but this structure can easily get destroyed when accessing stuff with traditional filesystem tools. As traditional links are unidirectional, it would either be very inefficient to figure out the links in the reverse direction, or it would require redundant links in both directions, which need to be kept consistent.

An alternative is to extend the standard filesystem mechanisms. This would allow to attach all necessary information to the files themself, instead of putting them as additional items in a special directory. Various modifications to the linking mechanism are necessary for that: Normally, links point from a directory entry to a file or another directory. For DeepaMehta-like relationships, we'd need links that directly connect one file to another. Also, they would need to be bidirectional, and have a type attribute. Maps could be represented as special filesystem entities, which can be browsed like directories, but with a map-like instead of the traditional hierarchical structure; presenting an alternative view of the main directory tree.

This way, navigating through maps should work in a pretty nice manner even with traditional applications; and when accessing objects with normal programs, they behave just like ordinary files. On the other hand, there is no way to access the object relationships and other additional attributes, or to modify maps, with tools not aware of the new features. Also, this approach is very invasive in general, making quite fundamental changes to various aspects of the filesystem functionality.

I think a more hurdish approach is to introduce a Mehta-translator: If you want some file to appear as a DeepaMehta-like object, just set the translator on it. Without the translator, it appears like a normal file; but with the translator set, it is presented as a pseudo-directory, with all the additional information. This way, the main file can be easily accessed and managed with normal applications (without the translator active), but the additional properties can also be represented in such a way that they can be accessed with traditional tools. Unlike with the directory representation on a traditional filesystem described above, the translator can enforce consistency when properties are manipulated through the pseudo-directory. It can also introduce other special semantics for the pseudo-filesystem entities, making it more intuitive.

Maps can be implemented with another translator, which creates a pseudo-filesystem representing the map as a special directory structure. (Much like in the variant with special FS features, but through a simple translator instead of modifications in the normal FS.)

8 comments:

Anonymous said...

This is great but everything gets clogged up once the map gets large enough to be meaningful. I don't know if you've looked at Topicscape (topicscape.com) have you? It works in 3D and allows many more topics to be associated with one another. The 3D landscape view lets you see much more and you can fly around the landscape and zoom in and out.

antrik said...

Well, I've never used DeepaMehta myself, so I can't really tell. But the idea is that in a map, you always only visualize these parts that you actually need for the task at hand; ant this should be quite manageable.

As I mentioned in the article, for some tasks the static map approach seems inconvenient, and some more dynamic navigation method could be necessary. Some sort of zooming mechanism might actually be helpful here. I don't see the benefit of a 3D representation over a simple zoomable 2D representation, though...

Also, judging from the screenshots, it seems that Topicscape uses a hierarchical structure and a fixed layout. This however is not really appropriate for most situations. The goal of DeepaMehta's navigation approach is precisely to replace unnatural fixed hierarchies by arbitrary associations and a free map layout.

Anonymous said...

I know this is an old post, but I'm intrigued. I took a brief look at DeepaMehta and blah... stupid java is horrible. But the idea is cool. And your comments about using Hurd style translators and an extendable filesystem are great.

I was thinking about user interface and the potentially overwhelming amount of relationships betwene objects... what about a dasher like interface? Not one that looks like dasher, which is pretty ugly IMO, but one that acts like dasher -- revealing options as they develop.

So for example, a file-picker dialog could have have a list of recently used files in one pane for quick access and then another pane with a text entry widget and then a display of relations that match. The user begins to enter some keyword that relates to whatever they're searching for. The display adjusts in real-time providing the closest fit options based on whats typed so far... once the user gets close to, or narrows down the options, a keystroke highlights the desired relation and the process starts over with either more relations to narrow down further or actual files if you've got it narrowed down enough. So the user doesn't need to know the filename, just have an idea of what the thing is and how it relates to what they're currently doing.... hmmm... letter... utility bills... overcharge... got it, January 12 2006 letter to electric company about overcharge and desire for a refund... make sense?

antrik said...

I didn't know dasher so far, but it looks interesting -- thanks for the link! I generally believe that in many cases a more dynamic interface can feel more friendly and be more efficient than traditional static GUIs. (Though one needs to be careful not to overdo it -- it can be distracting as well...) Dasher takes that to extreme.

Regarding your proposal, there seem to be two ideas in there. One is for the file chooser dialog instead of being limited to traditonal filesystem navigation, to have some kind of integrated search/advanced navigation facility. I guess this can indeed improve file choosers; however, I believe file choosers are a fundamentally flawed concept. As explained in the article in reference to DeepaMehta, the navigation facility should rather be the main view; and only once an object to act upon has been selected there, you launch some action on it -- rather than first running an action and only later selecting the object in a file chooser.

Which leads back to the main question, about efficient filesystem/object navigation. Your suggestion sounds like some kind of incremental search facility, where you narrow down the search bit by bit, giving more and more criteria. I don't know whether this is new (never worked with beagle and the likes), but it sounds like a good idea in any case.

Whether to use a zooming interface like dasher, or some more traditional kind of interface for that, I'm not sure. I said that I'd like some more dynamic way to navigate DeepaMehta-like relations, and some kind of zooming interface sounds like an interesting approach to explore...

Anonymous said...

I'm just reading this now, but I think you could be interested in the concepts behind Étoilé [http://etoileos.com/]. They combine some concepts from DeepaMehta and they approach them in a manner more similar to (what I think) you describe.

PS: Sadly, looking at the website they don't seem to describe the most interesting concepts in much detail. You can probably find more about that in the beginnings of the etoile-discuss ML.

Ikem said...

@antrik "I've never used DeepaMehta myself, so I can't tell."

There's a demo: http://www.deepamehta.de/install/client/demos.html

Ikem said...

@antrik "to extend the standard filesystem mechanisms [..] would allow to attach all necessary information to the files themself, instead of putting them as additional items in a special directory.

I think, a lot of what you're asking for is doable with extended file attributes: http://en.wikipedia.org/wiki/Extended_file_attributes#Linux

antrik said...

Well, that comment is quite on the spot: I originally meant to follow up this post with one about extended attributes. (But never got around to it...)

My Position on xattrs is rather ambivalent: On one Hand, they seem a good fit for storing the extra information used by the Mehta-translator and similar use cases. On the other hand, they are rather intransparent and awkward to use.

Ideally, we would have a special Hurd filesystem, which can store hierarchical data in an efficient native representation, and present it upon demand either as a filesystem tree; in a serialized form (as an XML file or tarball); and perhaps also in special representations like main file+xattrs.

Of course we would still have to consider how to represent the Mehta-Objects when saving them on a traditional filesystem for compatibility with other operating systems...