Regarding the last post: Thinking about it a bit more, my statement that this is unrelated to the issues I'm usually talking about here, is not quite true.
On the contrary: "-Ofun" is a nice way of putting it; but a large amount of the stuff dicussed in the essay (maybe even all of it), really boils down to tearing down entry barriers -- here, in a social context. Just shows how crucial this principle is in general :-)
This is somewhat unrelated to the technical issues I'm usually discussing here. But I'm agreeing so fully, and the issue seems so crucially important to me, that I want to point to it anyways.
Eric S. Raymond pointed out many organisational issues for successfull free software projects in his famous Cathedral and Bazaar paper. Most of the things he mentions there are right. But all of them are actually only a function of the fundamental idea the -Ofun, pointed out in this article.
And I know no other project so badly following it, so exactly doing the opposite of -Ofun, like the Hurd :-( It's really that, and nothing else, what makes Hurd's progress so slow. Can it be fixed? I hope so...
After recently commenting on KDE, I've now also stumbled upon GNOME's project Topaz, which -- together with some other pages it links -- describes future ideas for GNOME.
The Focus is a bit different than KDE's. (Fundamental UI changes, including "extensions", are mentioned at the setout, but there are very little actual ideas. Most stuff revolves around under-the-hood technical improvements.) However, with regard to new ideas, it's just as disappointing.
There are a couple of quite fundametal changes suggested -- mostly not terribly new, but still. There doesn't seem to be any special emphasize on any particular one of these. I'm just picking the one most relevant in my opinion: The VFS.
There is a page on the new GNOME VFS, which in turn links to the Desktop VFS from freedesktop.org. They are both describing mostly the same ideas, and I'm not sure about the exact relation between those projects, so I'll just treat them as one.
So they have -- correctly -- discovered, that POSIX file handling is unsuitable for most of today's applications. Full agreement here: We made the same discovery when designing various Hurd translators. open(), read()/write(), close() were sufficient in times when most Unix tools worked as filters, sequentially reading input files, sequentially writing to output files. Most of today's applications require different semantics.
One very important operation, as the GNOME folks have accurately observed, is atomically reading or writing a whole file. They want it because most of today's applications read the whole file when "loading" a document, and store the whole file when "saving" -- which is wrong IMHO, but that's a different story :-)
Nonetheless, in the Hurd it is probably even more fundamental: When a translator is exporting data through file nodes, it is extremely common for clients to read the node contents into a string, or store a string in the node. A simple operation doing this in one call, would be awfully useful. Not only because both client and server need considerably less handling for that, but also because knowing that the client just wants to write the whole file, is very important information for the server.
Generally, we need much more semantics in file operations than POSIX offers. Think of inserting data in the middle of a file. With POSIX, the only way to do that, is to overwrite the entire rest of the file. This is not only complicated and terribly inefficient: In case the underlying file is served by some more interesting translator, it can actually pose serious functional problems, if parts of the file are overwritten for no real purpose.
For operations that don't fit any of the generic semantics (write entire file, insert data, ...), we probably need to introduce transactions, to allow manually grouping primitives into semantic units. (This is probably what OGI's comment to an older post was referring to -- which would mean that at last I've understood the idea behind this comment :-) ) For many translators, it's crucial to know whether an operation is completed and data should be processed, written to the store/network, whatever; or whether following calls will alter it further. With POSIX only, some translators can only be usefully implemented by employing quite sophicticated caching and heuristics, if at all.
So, the GNOMEs are right about the necessity of new filesystem semantics -- though I don't know if they'll get all the issues mentioned above right. Sadly, that's where sanity stops in their proposal(s). A special filesystem API for desktop use only? That's absurd. How did they get that silly notion, that the file access requirements of "desktop" applications are fundamentally different from command line tools or daemons, so much as to warrant a special API for desktop use alone?
Oh well, I guess that's the general problem of GNOME (and KDE): Considering the underlying system(s) as given, they tend to pile layers on layers of workarounds, instead of much more simply and usefully fix it right at the system core level. (Reminds me of MS Windows, which started as a desktop environment, and ended up being an OS... In just a few years more, we will probably hear people say: "GNU/Linux? Isn't that obsolete? I'm using GNOME!") This just shows that we really need a GNU kernel, so the developers of a GNU desktop environment won't need to sink tremendous amounts of time into working around limitations of systems they have to run on in lack of a native one, where they could get all the functionality they need... But well, that's a different rant.
So, now that we agree ;-) an extra API for better FS access is silly, what is the alternative? That's obvious: Just like we already have Hurd extensions to POSIX interfaces in many other places, we should try to extend the standard POSIX file operations with the stuff we need, without forsaking compatibility. I'm pretty confident we can do this. (I've already discussed some aspects in conjuction with device drivers.)
Clients not aware of the new semantics, can continue using the old ones. Those that want to use the new features, will check with the server whether it implements them, and fallback to the traditional stuff otherwise. Most of this can probably be handled transparently in libc (client side) and/or the FS server helper libs: If a particular server doesn't know about the atomic file read/write operations for example, it will just get a series of standard POSIX requests doing the job instead. No need to force a switch to an incompatible new API.
