« Reconsidering Storage As A Service | Main | A Starter Private Cloud For IT »

August 21, 2009

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451be8f69e20120a5642ca8970c

Listed below are links to weblogs that reference The Future Doesn't Have A File System:

Comments

Matt Simmons

Taking the widest-open view possible, the only absolute requirements for the future of data storage are that it

A) holds data
B) makes data retrieval possible

You know as well as I do, I'm sure, that A doesn't imply B. I would hazard a guess that the hierarchical form of nearly every filesystem ever created has worked for decades because it simplifies finding data.

Psychologically, we arrange things hierarchically because of our evolutionary history. We understand hierarchy on a very guttural level. No one needs to be taught "how" a hierarchy works; we see a graphical representation of a tree and we "know".

The problems that you mention, specifically the trillions of files example, can be solved either through improving the existing paradigm so that it can handle trillions of files or throwing away the wheel and inventing a rocket ship. We all know which is more interesting :-)

Looking at various solutions that IT has come up with to organize widgets in a non-hierarchical format, tagging seems to be the mosts pervasive. I'm not certain of an efficient way to tag and then retrieve trillions of items in a non-hierarchical format, either, though. Maybe it's possible.

I suppose it's possible that the idea of what we're doing is right, but maybe we're looking at it wrong. If you look at a representation of a current filesystem hierarchy, it's two dimensional. Any particular folder has a parent, and possibly it has children. Maybe the future calls for multidimensional hierarchies? It's not hard to imagine the metadata as being represented by a third-dimension extruded toward the viewer, and there's no reason that it has to be limited to only three dimensions. Just because our disks are two dimensional doesn't mean that our storage model has to be.

Denton Gentry

Do you consider Facebook's Haystack implementation to be a forerunner of where the market will move to? An appliance which maps what is essentially a GUID to blocks on the disk, with no notion of directories and hierarchy? Haystack is awfully specialized for Facebook's needs, as it responds more like a web server than a regular file server, but perhaps we're moving to a world where all data is stored in things which act like web services.

Similarly though the original Google File System stored its data chunks in a Linux ext2 filesystem, I suspect the GFSv2 implementation will go directly to the raw block device. A hierarchal directory structure doesn't bring much benefit in that service that I can see, and the rest of the block device management which the filesystem handles doesn't seem worth it.

Chuck Hollis

Hi Denton

Hadn't heard about Haystack, but -- yes -- given your description, I'd consider that in the same vein. I'd be interested in how much metadata is associated with GUID, and what they can do with it.

Writing chunks directly to raw blocks isn't all it's cracked up to be. Disks can act erratically, and so on. It's all about useful abstractions, at least in my way of thinking, and the traditional filesystem is way past its prime for our current era.

Thanks for writing.

Steve C

Hadoop and Haystack and GFS are doing these things.

Really, it's not necessary to declare X dead where you sat around one day and mentally increased the order of magnitude of some critical part of X to where X falls over. In my head I can make every technology ever invented obsolete just sitting here on my couch. It's not an interesting exercise.

What is interesting is having a good appreciation for the strengths and weaknesses of various tools in your toolbox and develop good judgement about how they ought to be employed. Readjust slightly as time passes. Incorporate new tools into your toolset.

Statements like "Why are traditional file systems rapidly becoming obsolete?" are not helpful...what is that even supposed to mean? *nix-based systems all run on filesystems and that accounts for just about all internet servers (for instance). I don't hear of the Linux crowd moving away from filesystems (making better filesystems, yes), I don't see any new OS on the horizon that's going to change this.

Chuck Hollis

Hi Steve

This is not some idle mental exercise, it's a direct result of the work I do at EMC, including many real-world customer user cases that force the issue.

Sure, there are plenty of use cases that are just fine with traditional filesystems, no argument there.

Then again, if you prefer not to look forward, feel free to stare into the rearview mirror.

-- Chuck

S

'if you prefer not to look forward, feel free to stare into the rearview mirror.'

This is a false choice, quite obviously - your "visionary" reasoning/writing style demands that the future be set against the past, but that's just not a useful framing. You can use Haystack and be happy because you solved your scale problem. I hope you put something like this in place and I hope your real-world customers whose problems no doubt necessitate something like this are happy with the result. You can - even at the same time - use filesystems and be happy because you solved your everyday simple 'where do I put these few files in a sensibly organized way' problem - the number of successful, important software programs that need the latter will dwarf ones that need the former for at least several decades.

shiningarts

It is real interesting to talk about the future of file systems in general. Although the card reader and “vi” analogies seem appealing, in my opinion, they are not completely analogous to the file system discussion. As pointed out by various comments, we can understand the hierarchical file system because we perceive that nature is hierarchical just like the tree branches. However, come to think of it, the tree is not actually hierarchical at all; nevertheless, the tree knows what branches are obsolete and what leaves are active, which need water and nutrients to keep them alive and which are already dead. That’s not hierarchical, but we simplify it into a hierarchical structure because that is what we can comprehend.

EMC Documentum had been using unique ObjectID/GUID and metadata since its inception though. Speaking of metadata, the industry is still tagging it manually for taxonomy or classification which ironically defeats the very purpose why we are using the computer. Why can’t we make the computer review and tag the metadata for that purpose? Though it seems simple enough, we can’t do it automatically still. If we can’t automate such a seemingly simple task, how in the world can we understand such things complex things as trees in their actual state? I think the very technology we are using imposes the limitation.

Silicon based microchips are very good for processing the hierarchical structure, however, when it comes to organic matter, it easily crumbles down and can’t not handle it. Over time, we have invented greater abstraction and virtualization mechanism to overcome the architectural shortcomings of the current technology. Oddly, what the silicon based technology does real good is to present it cosmetically whether it’s to enhance a woman’s physique or whether it is some kind of Microsoft product. Both will leak and cause greater systemic problems at some point. The up-and-coming cloud computing is the mother-of-all abstractions that may be the last phase of the silicon based microchip technology. In due course, there will be some radical departure from the silicon based microchip technology that will help us to overcome the shortcomings of the current technology, and lead us into the greener pasture.

Alex McDonald

Interesting but flawed argument. The problem has been fixed for years with URNs. Read up on it at http://tools.ietf.org/html/rfc1737.

File systems only imply location because you (not the system) are applying some semantic knowledge to the name that the underlying system may not have at all.

The confusion you are making is between two things; one, how you organise and store the object (its location), and two, how you name the object.

This;

{3F2504E0-4F89-11D3-9A0C-0305E82C3301}

could very well be a hash of this;

/home/alex/myfiles/docs/archive/may2001/veryold.pdf

Same difference. A file is an object, just with a human parseable name, and a touch of metadata that we think of as hierarchical. It says nothing about where it is.

Chuck Hollis

@Alex

Some of view this problem very differently.

In your example above, if the underlying file were to move, the hash would no longer be valid, yes?

Also, there's no simple mechanism with this approach to associate significant metadata with the object itself, rather than a hash derivative.

Other commenters suggested "well, we've got that with Hadoop and Haystack". No, we don't. Hadoop (as I understand it) has no mechanism for object metadata (where it came from, what it should be used for, what it can't be used for, etc.) that's associated with data elements. Haystack (again, as I understand it) has the same sort of limitation.

Some will argue "well, we'll just build another layer on top of Linux file systems", and -- to a certain extent -- that line of argument has merit. Linux brings reliable I/O drivers and a cut-down kernel that many of us can live with, but not an object oriented information store.

I hope we can keep this discussion alive. Val seems to be coincidentally dropping hints that WAFL will offer an object interface at some point in the future.

It'll be interesting to see just how rich these objects will be!

-- Chuck

Alex McDonald

@Chuck

I blogged here on it and expanded my ideas; http://blogs.netapp.com/shadeofblue/2009/08/im-sending-chuck-a-bluey.html

No, it wouldn't change necessarily if it were to move. Let's take an example that I use on my blog entry;

http://blogs.netapp.com/shadeofblue/2009/08/poetry-corner.html


This file doesn't exist until you ask for it, because it's dynamically generated from parts. There's no directory shadeofblue, 2009, or 08, and no file poetry-corner.html either. And it doesn't live on a server at netapp.com. It's all name and no location, and it works across the entire internet, not just inside a single object store. It can move and nothing changes apart from the location where it's stored.

All through the magic of DNS and web servers.

Or, alteratively, let's generate a persistent and unique file handle from the file. An object ID if you like, for that's all a file handle is. Here's one;

blogs.netapp.com/shadeofblue/2009/08/poetry-corner.html

Same thing, just without the protocol. Its name and its location are two separate issues, and object-ids don't fix the problem; they just make it worse when you have to find the object. Try finding this;

c2f41010-65b3-11d1-a29f-00aa00c14882

You need an internet-wide object-to-location store to resolve the name to its location. Not good, because it has no hierarchy, bottlenecks around a single authoratative store, and it breaks all of the internet.

Chuck Hollis

Hi Alex

I can understand why you might think that way, but I'd encourage you (as well as everyone else) to take a deeper view.

I think you're focusing on the *presentation* of information objects, rather than the *objects* themselves. Sure, we're all familiar with different presentations of information (e.g. web pages) that abstract out physical locations, etc.

But the information objects themselves still live in a physical location, and have very little metadata associated with them.

Back to your example, web page "link rot" is an example of information moving or being deleted, and the upper-level protocols being forced to return an error, e.g. "link not found".

Furthermore, any metadata, services or policies associated with the information is now the responsibility of the application developer (or web page designer). There's nothing that is usually done at the information layer.

I'm sure many people will continue down the existing road of throwing more layers on top of something that's architecturally end-of-road as you suggest, e.g. let's just create yet another layer of abstraction.

Others will realize that something architectural will be done, and will be drawn to the "object-based information stores" as described here.

We know they work, and that certain customers want them, e.g. the popularity of Centera previously, and now Atmos. The question in my mind is -- when do these sorts of architectural approaches become mainstream in the thinking, and less of a special use case?

Note that Kotsadis' "DAS thinking" is reflected in each product's architecture already :-) And Atmos already has that "internet-wide object-to-location store" as an integral part of its architecture. Without breaking all of the internet.

Welcome to the discussion.

Thanks for writing.

-- Chuck

Arlo White

I agree with the general sentiment of the article. Most hierarchies are inherently flawed due to cross-cutting concerns. (Maybe a misuse of the term, but I think it works) If I'm organizing my pictures, I might choose date as the primary dimension and create a folder structure like:
2009/August/Vision Scuba Trip

But what if I want to view all pictures of a certain person, or all my Scuba Diving pictures. These are separate hierarchies that cut across my primary one. I could try creating these other directory hierarchies and symbolically linking to files from them, but this is cumbersome. Besides, you want a real database behind the scenes so you can efficiently sort and search your data.

You might say this problem is up to your photo software to solve. But I disagree. There is a need for a standard file database that is core to the operating system. Once you develop such a standard a whole new world of computing opens up. Suddenly you have the ability to create and explore data relationships in much more powerful ways.

Alex McDonald

Now to the second part of my long reply.

You say; "I think you're focusing on the *presentation* of information objects, rather than the *objects* themselves. Sure, we're all familiar with different presentations of information (e.g. web pages) that abstract out physical locations, etc."

You're making the common mistake of confusing http: (the protocol, or schema) with the markup language (HTML). What I showed was an http: request that returned HTML; it could have been anything at all. The receiver needs to be able to understand what is returned (it could be XML or an XML subset like SOAP for instance), but what I showed you was not presentation layer.

You say; "I'm sure many people will continue down the existing road of throwing more layers on top of something that's architecturally end-of-road as you suggest, e.g. let's just create yet another layer of abstraction."

Something like http: or ftp: schemas are very useful layers of abstraction indeed; they're what power the internet. A traditional filesystem is *only one* of many sources of data for a webserver. And I would place Atmos squarely in that traditional fileserver camp; it's a traditional, persistent filesystem that happens to use GUIDs or OIDs as names. (And Atmos supports SOAP, so you're doubly mistaken!)

You say; "Back to your example, web page "link rot" is an example of information moving or being deleted, and the upper-level protocols being forced to return an error, e.g. "link not found"."

That's addressed with relative, as opposed to absolute, URNs. They can be moved from directory to directory without losing addressability.

Absolute: http://blogs.netapp.com/shadeofblue/2009/08/poetry-corner.html
Relative: ../images/poetry.png

Anyone who uses absolute URNs gets what they deserve. Plus there's mapping; fuzzy, redirect, there are a huge set of options available. It definitely isn't fixed with object IDs either, which are at best "relatively absolute" (and my previous post about a request for an Atmos schema should make that plain).

You say; "But the information objects themselves still live in a physical location, and have very little metadata associated with them."

I say; that's because you insist on thinking of traditional filesystems, and then replacing them with another traditional filesystem; except it's got these object IDs and metadata.

Drop the "objects fix everything" attitude, and look at what we can do now, with remarkably little effort. And with remarkable flexibility; thinking outside the Atmos box lets you support varying types of objects. Things like

. transient objects with a limited lifespan, or number of uses
. dynamic objects that are programmable
. fragmentable objects where an object can be split into distinct parts
. modifiable objects that change in response external events (stock market data for instance)

...

There's a *whole host* of wonderful stuff that the Atmos can't do. Things have changed a lot since the 1970s. Modern filesystems as exemplified by web servers are very definitely not dead.

Paul Carpentier

Chuck: thanks for fingering "file systems" as fundamental thought obstacles when charting a future with x-illions of digital objects that may well have to live forever.

File systems focus on two things: storing and organizing. See the problem? Ask any photographer. You can't focus on two things at once, and focusing somewhere in the middle will only make things worse. Storage needs to deal with storage. Organization is the responsibility of databases and applications, so please don't get in the way with some hierarchical scheme that those disciplines have rightfully dumped somewhere in the mid-eighties.

Object storage is the logical answer to the hierarchical digital junk yard. Think of an object as a pod containing data and metadata. Its eternal, intergalactically unique identifier will allow to hook it up to any organizational schema(ta) that won't need to be reorganized every time you shuffle your server park. Its metadata will tell the system how to treat it (compliance, sleeping archive) and the applications how to find and categorize it (tags, keywords). Everything else is in the eye of the beholder: the application layer above it.

As opposed to hierarchical file systems, the object storage model lends itself extremely well to massively parallel implementations on networked commodity hardware, hence its intrinsic price/performance advantage vis à vis last century's big iron descendants.

It is vital to understand that the scope of this discussion so far is the actual architecture of the infrastructure. In other words, nothing to do with the user view of things, where the hierarchical presentation of files may live on for a very long time, simply because it's now familiar to nearly two billion people on this planet. As correctly pointed out in the post, this will only be a presentation model on top of a richer object store.

Which allows me to end this comment on a philosophical note: it is a sad truth in our geeky profession that whenever the Next Great Thing arrives in the market, it is immediately abused and coerced into a mediocre emulation of the Incumbent Dinosaur. When computers started to talk to one another, they transferred files using an RJE 2780 setup, with the sending computer emulating a ... punch card reader. When the PC happened to bring intelligence to every desk, among the first things it had to put up with was 3270 emulation: mimicking a dumb terminal!

So while we teach those exciting Object Storage clouds to mimic primitive file systems, let's indeed find consolation in the real and worthwhile applications that we know are right around the corner!

[ full disclosure: author of this comment was co-founder of FilePool, the technology foundation of Centera, and is now CTO at Caringo, bringing third-generation clustered object storage to the market in a pure software format ]

Chuck Hollis

@Paul -- good to hear from you!

Poor Alex over at NetApp seems to be fighting for his life in this discussion. I'm getting tired of explaining the thinking for the umpteenth time. I don't think he wants to hear it.

Congrats to you and the original FilePool team for doing truly innovative work at the time. Best wishes for success at Caringo!

-- Chuck

Alex McDonald

So much for your cheery "Hi Alex" and "Welcome to the discussion", and posting on my blog to say you'd responded here. And now your umpteenth explanation to me is making you too exhausted to respond. Or perhaps you tire easily and are short of fingers, as I can only see two replies to me.

Delete the other post from me that you failed to approve, as I won't bother posting here again. All I seem to get in response from you are sneering personal remarks.

Chuck Hollis

Hi Alex

I've posted all your comments here, so I don't know about the "other post" you are referring to. Other people occasionally lose comments as well, so my apologies for the inconvenience.

There's not much I can do about that, sorry.

I could tell from your commentary that you had little motivation to look at an alternate point of view, probably due to the fact that you're a member of NetApp's competitive team, and you've got a job to do, e.g. refute anything that EMC says vigorously and repeatedly. That sort of default positioning makes for a tiresome discussion.

I've gone back over my comments here, and I don't see any "sneering personal remarks". Maybe you can point them out to me? I don't see them, so sorry for that.

I was quite sincere when I said "welcome to the discussion". It's nice to see other vendors finally start to see some of the advantages of object-oriented storage. Between you, Val and Kostadis, there's a mini-renaissance going on over at NetApp :-)

We've had to explain our thinking on object information stores for many years, as well as position DAS as a suitable choice for the underlying media. As a result, we're pretty well versed as to where traditional file systems fit, and where they don't.

So, once again, welcome to the discussion.

Best regards ...

-- Chuck

Alex McDonald

Chuck

OK, let's start again. A little technical, but perhaps you can try again for the umpteenth+1 time. Here's the comment that went walkies.

An object has a name. Now, EMC have chosen, with both Centera and Atmos to use an opaque construct, (as Paul Carpentier calls it) an intergalactically unique identifier. It is just a name, no more and no less. It tells you nothing about the location of the "object". It's a URI, for which some explanation is required.

In non-technical parlance; a URI (Uniform Resource Identifier) is a string of characters used to identify or name a resource on the Internet. It's made up of two parts; a URN (Uniform Resource Name) is like a person's name, while the URL (Uniform Resource Locator) is where that person is; their address, if you like. The URN is used to privide both the URI (the identification) and the URL (the location).

Here's a "traditional" URI;

http://blogs.netapp.com/shadeofblue/2009/08/poetry-corner.html

Now we need to resolve the URI; in other words, we need a mechanism to find where the named data is located. In URI parlance, I need to dereference it and generate a URL for it.

The http: resolver splits the name into two parts; the DNS part (name to 32 bit (IPv4) or 64 bit (IPv6) ID). That's the URL. It does the lookup, and sends the rest of the string (shadeofblue/2009/08/poetry-corner.html) as a GET request to whatever webserver is at the end of that IP address.

Thet's the next part of the http: resolver; the webserver. It may use the passed name as a URL, in which case we could use a hierarchical file system to return a file of the same name in a directory structure.

But *it can do what it pleases* with that string. In the case shown, it composes a web page from fragments and sends it back to you. It could run a program, treat part of it as parameters, pass part or all of it on to another web server; it can do anything it cares with that string.

In other words, parts of the URN are URI (identification, the handle) and parts are URL (the location). What is what is entirely up to the resolvers. It can be *all* URN or *all* URL, or a mixture of the two, or both simultaneously.

So, let's find data on my Atmos. The Atmos URI is all name; the object ID is location free. So how do I go about locating the data from an "intergalactically unique identifier"? What might such a resolver look like?

If it's like this;

atmos://my.atmos.box/c2f41010-65b3-11d1-a29f-00aa00c14882

then all you've done is create a huge flat filesystem that has a URN or intergalactically unique identifier sitting in front of a URL that's located in a *private universe* at the end of an IP wire. It looks to me like NFSv4.1 (which supports metadata), but without directories.

That, to me, doesn't seem to be a great advance on where we are right now.

Chuck Hollis

Alex

If we look at name resolution (just one of the issues I raised), it's an architectural decision where best to put this function.

Putting it higher in the stack exposes the architecture to the potential that underlying things can move without informing the resolver, e.g. page not found.

Your example of "blueys" from the UK military was illustrative. The UK military knows where its people are at all times, their roles, etc. Nobody moves around without informing "the resolver". The UK military has real-time metadata for its soldiers that looks nothing like a file system or DNS.

However, it knows far less about non-military personnel, and if you gave the letter to the UK postal service, you'd get a different result.

Your argument is shared by many: that we can use internet constructs to create the appearance of location independence. And many people will continue in this direction.

But there's no arguing that there are many use cases that expect this location independence as an underlying service, and not as a presentation exercise.

If you go back to the original post, you'll notice that location independence (and dynamically keeping up with changing locations!) is only one part of the discussion.

There's the more fundamental problem of associating metadata with information objects. Take your "mypoems.pdf" example. Not much metadata associated with that puppy, is there?

Do we know if it should be dynamically cached in different locations to deliver acceptable service levels? Do we know if it needs to be archived forever on the world's cheapest media? Do we know if it is subject to compliance or security concerns? Do we know if it's associated with a specific applicaiton or business process?

Etc. etc. etc.

All of that can be "solved" by building YAL (yet another layer) on top of the legacy, or -- alternatively -- very cleanly solved by just revisiting the fundamental notion of an information store, drop the file system paradigm, and move to an object-based information storage concept.

Many people try to associate metadata with objects as an external construct: repository, metadata servers, etc. All are incredibly difficult and produce less-than-satisfactory results in many environments.

One could argue that -- persuasively -- metadata ought to be directly bound to the information object itself.

Solve those two problems, and you've set the stage for an even more useful discussion: uniform policies and services, directly associated with information objects, that don't depend on the presentation layer or application.

And that becomes a very interesting discussion indeed.

Another way of framing this whole thing might be "what will file systems look like in the future?". We may choose to call object-based information stores "file systems" out of simple convenience.

But they won't look much like the file systems we use today, will they?

We get routinely criticized at EMC by you and others for all the diversity in our product portfolio. We have storage products that stores information in blocks, or in files, or in objects.

We also have a variety of agnostic metadata layers (e.g Documentum) that solve aspects this problem in a more traditional manner.

One positive of this diversity is that we have the freedom to think about information storage problems differently, and not be locked into a legacy thinking pattern, which -- frankly speaking -- I see you and the other NetApp folks struggling with. Hence Centera, and now Atmos, and soon a bunch of other cool things as well.

Hope this helps.

-- Chuck

Alex McDonald

Thanks for the speedy response. Yes, I do understand where you're trying to go with this. My argument is not your argument. I fully accept that traditional filesystems can't do what we need them to do. Hence my example of a webserver; it's moved on in terms of its flexibility in serving up named data. I really don't care that it's an object ID or what looks like a filename. That's a red herring. The point is; it's not only a name, it finds my data too.

What we are clear on, and where we're at cross purposes, is that we need some form of directory, "resolver", or ID-to-location service if all you have is an "intergalactically unique identifier" that's got no information as to location. The question still stands; what and where is this ID-to-location service in the Atmos solution?

Internal? Can I address an Atmos and have it find any object by ID for me, across all the places I may have stored my data? Today I may have a single object store. Tomorrow, I may want to amalgamate that with an ISPs data store. I may want to build a global object store in collaboration. How do I prevent bottlenecks, provide parallelism and get at my data in this case?

External? Do I need to run or have access to, along with everyone else on the planet with whom I wish to share data, a global ID-to-location directory service?

DNS does much of this; it maps names to object IDs, except we currently call them IP addresses. But it's hierarchical and devolved, because that's the best way we have found so far to provide scalability, parallelism, maintainability, routing and sufficent robustness to name a few of the problems it solves. It works globally, not just in my datacenter. What ID-to-location service is EMC suggesting be used instead?

I hope that this is clearer. It's not legacy thinking, btw; it's amazement that you are suggesting that not only hierarchical filesystems are dead, but that name spaces based on hierarchies are dead, that the very structure of the internet needs to change.

If that's your position, I admire your cojones.

Chuck Hollis

Alex

Interesting point of view. BTW, your competitive agenda given your role at NetApp tends to make any sort of productive discussion with you extremely difficult.

For example, if you go back and read my original post, you'll notice that I never used the word "dead" (you did) or that I even suggested that internet-based protocols were dead or needed to change (you said that).

I guessed I missed the trick about putting words into someone's mouth, and then taking them to task for saying what they did :-)

The problem is I can't tell whether you sincerely don't understand, or your just pushing your usual fight-EMC-at-all-costs agenda. Which is it?

Fundamentally, we're arguing different sides of the problem. They're not mutually exclusive. Both work quite well together, I believe.

Give it some time -- the thinking will seep in, and you'll start to appreciate some of the attractiveness of this alternative view. And, BTW, we're finding more customer use cases where they agree as well.

So, on a related note, will someone explain to me how NetApp's announcement today had anything whatsoever to do with "cloud"?

I mean, there was a nice assortment of technology giblets and doodads in there, but I just couldn't connect the dots and get to the starting premise, e.g. "cloud".

I'll go re-read it all though. Maybe I'm just a bit slow.

Cheers!

-- Chuck

Alex McDonald

Chuck,

I could accept that "dead" was hyperbole (or a straw man argument) if your piece didn't start "The Future Doesn't Have A File System ... at least, not that tree-oriented, hierarchical model we've all gotten so accustomed to".

You have it shuffling of its mortal coil; I shorten it to dead. It's the same sentiment in fewer words.

My curiousity and disagreement has little to do with any competitive agenda; it's not as though we run into Atmos on a regular basis. I am genuinely interested in how EMC and others with object stores are going to square the internet circle.

If it's a good idea, I'll applaud and we'll copy (patents permitting). Right now, I suspect you don't know how to do the essential step of melding hierarchical location spaces with flat name spaces and making it even marginally better than what we have. That's disappointing.

On cloud, I'm no expert. Have you read Dave Hitz's blog, or Jay Kidd's?

http://blogs.netapp.com/dave/2009/08/three-cloud-strategies-every-cio-should-consider.html

http://blogs.netapp.com/jay/2009/08/you-really-do-know-clouds.html

Paul Carpentier

@Alex

As I checked out your blog wondering about the origins of the parallel universe non-discussion above, I stumbled on your Poetry Corner and immediately understood that this very likely provided the ideal medium to explain:

Mirror, mirror upon the wall
No intell'gent life down here at all?
Why would people want to see
their objects in a Christmas tree?

Mirror, mirror upon the wall
don't they understand at all
that their ways won't ever scale
as their b-tree schemes will fail?

A filename just can't take the heat
Unique IDs are what you need
That live forever and three days
avoiding '404' malaise

(To which the mirror replied:)

The NASty queen, though fair she be
has never been a match fer ye!

shiningarts

When we discuss file systems, we are not discussing the file alone. The discussion encompasses the process, presentation, communications, etc. Contrary to the belief of some here, the object oriented approach to computing or storage is the best mechanism out there so far considering that the current computing architecture is based on silicon. As pointed out by one of the commentators on this blog, the hierarchical display of the file system is merely one possible way of describing the presentation layers of the object-oriented file system architecture. Also, it is naturally a good way to separate the storage and presentation layers (database and application) because it can use the processing resources effectively.

OS systems such as Unix/Linux have been using hierarchical file architecture because it deals with more archaic and machine level processing traditionally. However, in due course, it will change as well. On the other hand, storage purveyors are banking on the hierarchical nature of this day and age; eventually they will encounter a brick wall. Speaking of brick walls, when one encounters such a wall, it’s not very effective to argue about what kind of brick we just hit--whether it was square or round--ad nauseam. We need to pay attention to what is more productive for us to address to further the discussion constructively.

Raphael Soeiro

Hi Chuck,

Nice post!! I've been reading you blog for about a year but this will be my first comment. :-)

Would you say that after we get rid of HardDisks-HDDs (in my opinion the oldest piece in modern IT) File Systems would be the next "piece" to be changed/replaced?
I like the idea of a non-location based architecture and would like to see this in the future.

Best,

Raphael

Alex McDonald

@Paul


Mirror, mirror on the wall
Where's my object I would call?
She gave a name I whispered sweet;
But no place giv'n so we could meet.

Mirror, mirror, bring her near!
I cannot live without her here.
But she has nought but a strange name,
No place she gave her hand to claim.

I used to think a 404
From other filenames was Web woe;
But her I need to make me whole
Her bits and bytes are in my soul.

Mirror, mirror on the wall
Can meta data help at all?
Or is her name all I can sound?
O mirror, please, I need her found!


To which the mirror replied;

The object queen, though fair she be,
Has heart that's cold to hierarchy.

Paul Carpentier

@shiningarts

I couldn't agree more. We all seem to be in agreement that we hit some wall, that it's hard and it hurts; and that indeed we need to further the discussion constructively. IMHO, in that respect, the most urgent item on the agenda would be to develop a common understanding and agreement on a minimal, straightforward way for applications to access cloud/object storage, regardless of the underlying vendor technology or implementation: an open standard, just like the web! Immediate and future consequences would be staggering indeed.

Yet the exercise is surprisingly straightforward. In order to further the discussion, I've tried to summarize my initial thinking below, just as a starting point for any volunteers to contribute to.

To keep things reasonable and within reach, the basic idea here is to describe functionality in levels, just like layers of an onion skin. Even the simple agreement to access stored content using a similar unique identifier ("Level 1" below) would be an invaluable step towards our children's application future. Supporting 1 through 4 i.e., "Level 4 support" would allow any application to read from any standard repository, etc. You get the idea.

1. Unique identifiers; 128 bit, hex representation proposed
2. Object = immutable [Content + Metadata]; content is free format, metadata is free format, XML recommended
3. Simple access protocol; HTTP proposed; non-normative client libraries optional
4. READ and READ METADATA operation; (READ gets metadata and content)
5. WRITE and DELETE operation
6. Small set of standardized XML policy metadata constructs re service level, compliance, life cycle; TBD
7. Persisted Distributed Hash Table to allow variable identifier mapping; 128 bit to 128 bit; HTTP accessed

Any thoughts welcome. Please feel free to join the effort!

@Chuck: my apologies if this looks like hijacking your blog page. Feel free to refuse or (better?) start a new topic!

Chuck Hollis

@Paul -- have at it! Great discussion ...

Preston de Guise

I've been debating whether or not to join in on this discussion, but I do want to make one point from a slightly different perspective.

While there's several things I disagree with Chuck on (most notably, Cloud computing), I do wholeheartedly agree that the traditional filesystem must die. Whether there's a true, viable replacement to the filesystem out there is debatable. I would argue that because the vast majority of people are still using filesystems in their traditional format, there's currently no viable contender. (I.e., the development of a viable contender will be a "eureka"/everyone switches scenario.)

Why do I think filesystems must die? Because I've been working in backup for 13+ years. Honestly, it appalls me that in all this time, the amount of effort put into performance improvements for sequential walks of entire filesystem hierarchies has been next to negligible. The filesystem vendors and developers have let the world down on this front. We keep on building bigger, more scaleable filesystems, but the entire focus of that development has been on random access improvement and ILM. Where's ILP catered for in all of this? This lack of improvement in sequential walk performance has led to the development of backup functionality that shouldn't need to exist - block level filesystem backup options. I.e., such functionality exists to solve a problem that shouldn't be there.

Michael Hay

Chuck, I started writing a response it turned long so I posted a response on my blog.

http://blogs.hds.com/michael/2009/08/a-response-to-chucks-thoughts-on-file-systems.html

David Slik

@Paul

Items 1 through 6 on your list above are almost identical to the work that SNIA has been doing on creating a standardized RESTful interface for cloud storage. I'd encourage you and everyone else that reads this blog to take a look what we're doing at SNIA in the cloud space, and get involved.

http://www.snia.org/cloud/

I'll respond in more detail on my blog after my flight.

Paul

@Chuck

I would like to hear more about the object store being talked about that will make the "tree-oriented, hierarchical model" filesystem obsolete.

Simplistically we can consider a traditional Unix filesystem a collection of objects. We have inodes and directory entries to give the data blocks structure and provide the metadata and relationship for the data blocks.

Obviously object-based information stores provide a way to store objects and handle a certain amount of metadata to track where the objects are physically stored, but where is other metadata about those objects stored?

This discussion has talked about having an object identifier (GUIDs have been presented several times) so I’ll use that here. I have 4 objects, represented by the following object identifiers:

{5059c2f4-3fef-41de-8556-43472dd2613d}
{6f9bb4bb-ae14-492a-b463-6059f11931b0}
{ff532375-7dcd-4e82-bbcb-edb403596742}
{1c986a67-329b-4b1f-bd77-1e4e41b94989}

Where do I store the metadata for those objects? While I might remember that the first is a list of recipes and the second is my calendar I’m not likely to remember that information for very long and then I’ve effectively lost that data.

Where does the object-based information store store its metadata about the objects, or does it?

I should point out, I’m not with another vendor, my interest is merely that of a customer who uses storage for everything.

Chuck Hollis

@Paul -- good question.

The key here is the desired abstraction -- do we want rich metadata to appear always associated with an object in such a way that the object and metadata are always consistent, can be never separated?

With a traditional filesystem, there's no mechanism for doing this other than creating a higher level abstraction, or the proverbial "bag on the side". Both are unsatisfactory in many use cases, but not all.

When I write an object (file) I want to add as much metadata as I can. When I read an object I want the same.

Going further, it's be great to have other processes add metadata as needed. And it'd be great to have other processes (e.g. backup, compliance, search) use that metadata as well.

Back to your example, imagine that when you said "PUT" to an object, you offered up a pointer to the object you'd like stored, and also offered up some tag/value pairs that represented the metadata you cared about.

You'd get back one of those content identifiers as you suggested, and that's it. To get to your data, you either supply the content identifier, or you search on a metadata tag of interest.

So, if that's what ideal might look like, what's underneath the covers of this object/metadata abstraction? There's where the innovation can occur.

For example, some implementations of object storage use lots of servers with DAS, run a little Linux filesystem on top of it to get some low-level services, and then write the software layer that implements all the magic I just described above it.

The user of the object storage isn't aware of any of that, though. They don't see the underlying filesystem or database that might be used to actually store the objects and the metadata.

Just like we don't expose machine registers to applications any more, the trick is creating a useful upper-level abstraction that solves the problem at hand.

Did I help here, or just make matters worse?

-- Chuck

shiningarts

I completely agree with the notion that "The Future Doesn't Have A File System." A couple days ago, when I was checking EMC Documentum system in our installation, I noticed an inconsistency in the system area; by correcting the obvious issues that were causing the problem I was encountering, I actually corrected more than I could have ever thought of. This was possible because Documentum was built on the object oriented concept from its inception. The Object Oriented mechanism is one of the best ways to encapsulate our known world with our contemporary silicon-based computing technology. In my own experience, when I design an application, I can’t completely encapsulate all of use case possibilities before releasing it to the user community. The Object Oriented mechanism helps to correct mistakes that developers overlook because it closely parallels Mother Nature.

Look around at the natural objects that you see near you. You can discern what it is because it displays its content naturally with the shape, smell, color, or whatever it can able to convey to help you to identify. Nature, to put it another way, is object oriented and encoded with metadata. I can tell something is an apple without eating it to verify. In nature, we don't need to actually take something apart to and examine physically, for the most part, because natural objects are coupled with naturally encoded metadata.

Now, try to figure out the file system. You can't tell what it is until you open it up. In some unlucky situations, you could unwittingly open Swine Flu and your computer would be infected and destroyed all together. Currently, we don’t use the intrinsic Object Oriented file system out there right now. Even Documentum has to create separate metadata pools in a database which points to the file system in a separate filestore. This definitely slows down the process unnecessarily. Wouldn't it be nice if we could just discern what the object/file is because it shows the metadata right there? Even better, just like the nature, if the object could self-reflect the metadata automatically without human intervention?

To speed up the machine processing with metadata, we could use barcodes instead of human language describing the metadata. If this were to really happen, the OS systems we have known our time, such as Unix, Mac, or Windows, would be history. It will be altogether different process to consume the intrinsic objects with self-reflected metadata. Eventually, and this is a really wild idea, we could have ocular implants, similar to cochlear implants, which would allow humans to directly process the metadata barcode without translation.

If this had already happened, EMC wouldn't need to acquire Kazeon after all. However, what I understand is that Kazeon melds or stitches well into the EMC CMA family. For more on this topic, take a look at this article:

http://industry.bnet.com/technology/10003275/emc-stitching-its-stack-with-kazeon/

twitter.com/udaysubbarayan

I agree on the need for "object store" as I myself building one for last 2 years.

But I disagree on future don't have a file system.

-UdayS.

paul moore

This is an interesting subject as I've spent the last 13 years in the content management field and this is exactly what we do.

Filesystems do what you describe anyways, they have an object and they map it down to the actual physical storage layer and then give a user interface to that information.

The old BeOS in the early 90's had alot of these ideas within in it, the ability to add metadata to a file and then be able to search on that metadata.

The key to all of this which I believe somebody stated, is how do you automate the gathering and applying of the metadata to the "file". This is what we are doing in the content management field, and it's the highest cost of storing information.

A file is nothing more than the encapsulation of information irregardless of what form it is in. It does no good until something either a process or a human takes that information and renders against it.

The current filesystem is the fastest way for somebody to "organize" information. I'm not saying it's the best, or doesn't have it's limitations. And organization and searching are two different things.

I agree that you can't take legacy code, except for one paradigm that I've seen and that is taking the metadata and creating file structures, that present via file system such as WebDav. This can allow a single object to show up in multiple places in the filesystem hierarchy.

Damir Lukic

Sorry if my idea of storing file is similar or equal to some of those mentioned in these comments, but I haven't red everything, just been thinking how to solve this problem.

Anyway, I'd lose everything that file is, and go with different approach. It's name is not important, nor is it's location. In fact, I'd use object that should be named after it's content via dictionary-based hash. So basically, when file is copied onto storage device, it's been fully indexed and the global dictionary is been updated with new unique 'words'. Global dictionary keeps all unique information from all files inside of it.

File is now presented as a set of links to global dictionary. This is some sort of dictionary-based compression.

But, the beauty comes later. :) What I wanted is to make files intuitively searchable. Storage device is able to intelligently index files. For example, it is possible to index pictures by the amount of Top5 colors inside it (let's say that pictures from vacation at the ocean holds a lot of blue colors, device is able to understand that the picture has got a lot of blue inside it, and can understand that this picture represents sea or sky making that picture a candidate for searches including 'sea', 'sky', 'blue', etc). Similar to other media files which are pretty bad to index (let's say mp3 files).

Intuitive search enables user (or machine) to search for files using parameters that are pretty fuzzy. Like a brain. Try to think as a human being. I want to find some classical music that is very similar to Chopin, or I want to see my pictures that I made last summer when being on vacation on countryside.

In multiuser enviroment, this is even better. We can introduce new term - I call it filegroup. If we make a uniqude identifier for every filegroup (an interlink between user and data) we can make a file (data in fact) a part of any filegroup we want, but don't need to copy it all around like we had to do it with file-folder structures. Yes, we have shortcuts, but it was way too complicated and uses a lot of space.

I believe that this idea won't be usable for at least next 10 years, but once flash drives and CPU power raises to levels usefull for AI (Startrek computing), we'll see. :)

The comments to this entry are closed.

Chuck Hollis


  • Chuck Hollis
    Chief Strategist, VMware SAS BU
    @chuckhollis

    Chuck has recently joined VMware in a new role, and is quite enthused!

    Previously, he was with EMC for 18 years, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Holliston, MA with his wife, three kids and four dogs when he's not travelling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not buy him a drink when there is a piano nearby.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!