Aristotle vs. flat indexing: why we chose tags instead of directories

The search for order has always been a human necessity. Only that which is regularly repeated makes it possible to predict a modicum of the future and loosens the perennial grip of chance. Laws of nature discovered through the scientific method are essentially recurring phenomena. Truth itself, according to Simon Baron-Cohen, is for us humans just something that repeats itself. As a neuropsychologist with expertise in autism, he has studied extensively the human types most capable of recognizing patterns and repetition around themselves. He identified the so-called “hyper-systemizers” in whose ranks we find the great scientists, inventors, mathematicians, and artists.

The origin of our method of organizing the world can be traced back to the first outstanding example of a hyper-systemizer and pattern-seeker in our history: Aristotle. His “categories” –substance, quantity, quality, relation, etc. – were the primary and supreme characters of everything, like a set of coordinates that taking specific values for each thing identified its place in the universal order. Categories aimed at sorting all that exists, and also at describing evolution by their variations.Aristotle’s aspiration for universal ordering never died, and his systematics has always found new adherents to renew its fire. In general, it proved to be beyond human capacity: in the nuanced and complex realm of everyday experience, Aristotelian universals were helpless, and every supposed rule faces innumerable possible exceptions. Wittgenstein softened their claims by reducing them to a simple “family feel,” a similarity between phenomena that can be guessed but not precisely described.

However, the method has been successfully applied in confined areas. In the modern era, all academic disciplines have produced their ontologies, that is, their particular “sciences of being” made up of categories, to structure knowledge and narrow complexity. A famous example is the taxonomy that Charles Linnaeus established in the natural sciences in the 1700s, starting with the genus + specific difference pair prescribed by Aristotle in Metaphysics. Linnaeus extended this basic hierarchy to increasingly refined levels that could encase all known living things. Being open-ended, it was modified and expanded over time and despite the centuries has remained a fundamental tool for organizing scientific knowledge.

Taxonomy has also been used effectively for cultural artifacts. For example, documents, the subject of archivistics. Which became a science in its own right in the 16th century, detaching from paleography and diplomatics to devote itself to the preservation of memory. The culmination of its efforts is the 1994 General International Standard Archival Description or ISAD(G), which defines the descriptors needed to organize and identify documents of all kinds in a consistent and comprehensive manner. Here, too, the descriptors are structured in hierarchical levels and proceed from the general to the particular.

Another notable case is books. In libraries at the beginning of the modern era, different institutions used different cataloging schemes: for example, at La Sorbonne in Paris, books were sorted first by subject, and then alphabetically. In the late 18th century came the invention of card catalogs: each card is a bibliographic record that virtually represents a particular physical work by means of a fixed set of data such as author, title, etc., called index terms (we will return to this later).

Conceptually, the card is a great step forward to our time: if you think about it, in fact, the resource descriptors contained in the card are nothing more than metadata. The transition to computational systems in fact keeps within the tradition, as far as knowledge organization is concerned: its computer version is a construct extremely familiar to all of us, the file system. The file system is the operating system component that segments an immense and indistinct mass of bits, assigning a name and certain properties – metadata, in fact – to each significant portion. Not surprisingly, the term “file” is the same one that was used for paper catalogs: clear evidence of the total conceptual continuity between analog and digital archiving. The continuity is also found in the information architecture: among the principles and devices inaugurated by the Multics operating system in 1964, which set the standard, is the hierarchical file system with the common directory tree.

Even the idea of the information layer superimposed on the physical world, which was already in bibliographic cards, is taken up and enhanced in computational systems to reduce inhuman complexity. What the user sees is decoupled from what the machine sees. «The basic structure of the file system is independent of machine considerations. Within a hierarchy of files, the user is aware only of symbolic addresses. All physical addressing of a multilevel complex of secondary storage devices is done by the file system, and is not seen by the user. (…) In general, a user will not know how or on what device a file is stored.»[1] This trick will also serve to present the user with a virtual memory space larger than the physical one, and allow programs to run that would not fit in the physical one, using a swap file.

Such tricks were repeated and extended by Ken Thompson from the earliest versions of Unix, where we find the mature version of the hierarchical file system as a tree of arbitrary depth. In Multics, directories were also “files”; in Unix this generalized file concept is extended to the device abstraction. In the new interpretation, directories are virtual containers that list only the names of files with their pointers to so-called inodes, which in turn contain file attributes and the addresses of the memory blocks where the data are located. Unix completely decoupled hierarchy information from file access information, and needed a numeric index to go from one to the other in the access table: hence the name i-node, short for index node.With this comes the possibility of having multiple filenames in different directories pointing to the same data, and the file system is able to map graphs onto the tree. Probably the innovation reflects the fact that the simple tree structure was already too tight, and the need was felt to multiply the ways of relating files to one another. Since the logical structure of files is machine-agnostic, how to organize them becomes a free choice of the designers.

Like programming languages, the logical interface layer has been refining and expanding abstraction capabilities more and more as processing power has grown. Whereas twenty years ago a single, coherent object could be in fact scattered in a lot of small data chunks on a disk, perhaps to optimize space or read/write operations, in today’s cloud infrastructures we can see as a single object what is distributed across many hosts on different continents. We can also run many virtual machines on the same physical system, and even virtualize operating systems themselves by running small parts of them in separate containers.

The current deluge of information, in which search engines are our indispensable guide, reminds us that the inseparable companion of preservation is information retrieval. What is archived should be retrievable and exhumed with reasonable ease, and for this to happen, techniques are needed that take into account both what ‘information looks like and the cognitive and psychological capacities of the humans who need it. The aforementioned ISAD(G) archival standard, for example, avowedly aims to «identify and illustrate the context and content of archival records in order to promote their accessibility», to foster the «intellectual control necessary to ensure that reliable, authentic, and meaningful descriptive records are transmitted over time», and to facilitate «the retrieval and exchange of information (…) in a unified information system».

Essential in information retrieval are indexes, what we have seen above as index terms in cards and as index nodes in Unix. In general, indexes are signs that, precisely, indicate other things. In C. S. Peirce’s semiotics they link signifier and signified in a relation of contiguity, especially of cause-and-effect: the footprints of an animal and the presence of the animal, smoke and fire. Thus the set of index terms that make up a bibliographic record stand for a particular book in the library. The index terms of the cards all together form the dictionary of bibliographic records, in practice they are the Aristotelian categories of that world: the parameters necessary and sufficient to catalog the universe of books in a physical library so that they can be found at a glance.

Those who used the first popular search engine, Altavista, in 1995 will remember that it kept a list of thematic directories on the left column: these were clear vestiges of the old cataloging method, which in that remote era was still thought to be enforceable on a Web made up of a handful of sites – only 23,500 according to NetCraft and Internet Live Stats’ estimate. The distance between Altavista’s homepage and the minimalism of today’s Google homepage tells better than any words the radical transformation that information retrieval has undergone: only one field for user input has survived. Because there are now some 2 billion sites out there, index terms are more than the stars in the sky, and the only thing that makes sense is to give users carte blanche. This freedom is assisted by suggestions that are also statistically derived from users’ search keys.

Top-down categories were already being challenged on another front: the new wave of the Web that arose in the early 2000s. The lazy interactivity of Web pages received a dizzying momentum thanks to UI innovations such as Asynchronous Javascript And XML (AJAX), and the user began to materialize as an active producer of choices, content, and conversations. It is the birth of what Tim O’Reilly christens Web 2.0, characterized more than anything else by the ubiquitous share button, and the tide of User-Generated Content starts to swell the Web unabashedly.

Grassroots mass participation makes entirely new stuff possible, in which the collective drive of distant and unknown people can be synchronized upon a single online goal and concretize into extraordinary results in terms of numbers. These are numbers of unprecedented magnitudes, these are the new mass phenomena to which we are accustomed today: for example, the many kinds of crowdsourcing, or the viral effect that can quickly take egregious strangers from oblivion to millions of ratings in a very short time. And the immense, atomized, talkative crowds of this superconductive global society are more impervious than ever to the old top-down methods of cataloging. So many people are interacting that it is impossible for any authority to claim exhaustive, universal ways of reading and sorting the world (except, of course, for Big Tech imposing their own categories through platform software).

Therefore bottom-up, user-chosen categories are born: tags. Aristotle must have turned in his grave to see ordinary people decide on favorite keywords to stick to their digital assets, to organize them, explore them, and retrieve them in their own ways. The idiosyncratic order of one’s bedroom extends to the entire Web 2.0. With tags, taxonomies are replaced by folksonomies: classification systems, as the composition of the neologism suggests, that put the spontaneity of the folk, the popular as opposed to the authoritative, the common sense of the people, in place of prefixed hierarchical taxa.

What is really new and remarkable is that tags are horizontally visible between users, link to each other and overlap, giving rise to unprecedented phenomena. For example, the miracles that Photosynth has been doing for a few years since 2008. It was a collaborative platform, now sadly gone, where anyone could upload travel photos with their good tags. Then an image-processing technology from Microsoft Research, most advanced for the time, would combine them based on their tags. It could recognize the commonalities between the scenes portrayed, taking into account different perspectives, and thereby calculate the three-dimensional scenery of which they were part. Stunning pointclouds, swarms of bright points that formed the 3D gestalt of a panorama or building, such as San Marco’s Basilica in Venice, were found on the site. These 3D models were navigable in the browser.

Even more remarkably, one could switch at any time from pointcloud divisionism to photographic visualization, to explore the location through its primary constituents: thousands of photos tagged “San Marco” or something. The synths showed photos in the right perspective relative to the current viewpoint in the browser. It was a truly unique intermedial experience. And a socio-technical experiment, too, considering that so many people had taken those shots in their own travels, never imagining they were building a collaborative work.

This collective horizon was already in the very first user-defined-tags-based service I remember using: the handy del.icio.us social bookmarking service, founded in 2003. Saving the most interesting sites in my online profile instead of the solitude of my browser, browsing other people’s tags and so discovering other worlds, was my first real experience of immediate horizontal expansion of knowledge, typical of the information society. What now, never meeting any resistance, has slipped into our normal information overload disorientation, a constant veil of nausea and performance anxiety.

User-defined tags have proven to be the index terms best suited to the global network information ecosystem, malleable enough to adapt to the enormous variety of sources and usage practices encountered around the world. It is the victory of the particular over the Aristotelian universal. By linking together, tags can compose frames of great complexity, which at large scales are spontaneously structured by the statistical properties of the whole. However, even in local contexts, their nonhierarchical modularity allows for the creation of graphs of interest, for example, operating on idiomatic dictionaries specific to a field of knowledge or work.

This is why in Mnemonica we adopted tags instead of the hierarchical file system when faced with the main design choice: how to organize uploaded content. In media asset management platforms of the audiovisual industry, they usually follow the classic file system scheme for organizing clips: a project contains folders, each folder houses content, on each folder you can do certain operations (e.g., sending to a set of recipients), on the contained files you can do other operations. These platforms are inspired by a principle of habit: because the hierarchical file system is so absolutely commonplace for everyone, it is what everyone expects to find in a file management interface, and everyone knows how to use it without instructions.

But even here the universal does not necessarily go along with the particular. Was the directory tree the most appropriate information structure for people working in film production, considering their specific workflows, the rapid times with which they have to access content, the kind of data they generate, the order in which they do so and then return to consult it, the way they interact with one another, and so on? Our answer was no. To us the hierarchical file system, as indeed the folder itself, seemed archaic and too rigid compared to the day-to-day needs of cloud cinema. That fluid method of work following a plan but continually adapting to circumstances deserved to be indulged by an equally flexible scaffold. And the perfect joint to build it was the tag.

So we designed our flat indexing, which enables some very special features of Mnemonica.

How does it work? For starters, the media are housed in Screening Rooms that are not folders but controlled social viewing spaces made specifically for the requirements of film/TV production. They are also active environments, in the sense that they are also capable of automatically processing their content, e.g. applying watermarks or distributing content to many separate audiences at different times.

Then in the Screening Rooms we have Days and Bins, which look like containers or folders but are neither. Both Days, corresponding to days in the production plan, and Bins, multi-purpose containers that can be used, for example, as scenes or as casting roles or locations, are actually tags associated with media. Other tags can be related to these as desired, while Bins can be associated with both generic tags and semantically marked tags such as location and character.

The simplicity with which they accomplish information retrieval, precisely because of the underlying tag structure, is truly unique. And here we should note something en passant: when people believe that the file system is the best thing because it can be used out-of-the-box, with nothing to learn, they forget that there is another case where the manual is not needed—when things are self-explanatory as you use them. That’s what happens in Memonica: when you tap on a Day and see it light up and at the same time see both the media pool and the Bin list next to it shrink, and then tap it again and see everything roll back, you instantly realize that that a Day works as a filter that selects in the media pool only the clips it contains. As anticipated, however, “contains” is just a convenient metaphor that aids UI learning: the real action of a Day is to select a tag over a set of media.

The same is true for Bins, whose behavior is equivalent. And even this very useful symmetry is possible only because Days and Bins are media tags and are not nested hierarchically, which would limit the possibilities. They are peer entities, connected through media. So to make a playlist I can select any Day first, and then one of the scenes shot on that Day; or, conversely, I can select any scene first, and then one of the Days on which it was shot. This effective immediacy is extended and further enhanced by the combined system of all the other tags that users can define in Mnemonica, and which are all filterable via the search fields scattered throughout the UI.

In our opinion, and according to our users nonetheless, this is the kind of elasticity in sorting content that best suits film people. Because #welovefilmmakers more than Aristotle did.

[1] R.C. Daley (MIT), P.G. Neumann (Bell Labs), *A General-Purpose File System For Secondary Storage *https://www.multicians.org/fjcc4.html