Monday Dec 01, 2008

Celeste - what is it?


In my travels of late Celeste has come up under several different circumstances. I'm going to attempt to collect a few details in a place to point people at and say how cool it is here, but in the end point you at the project.

Celeste is essentially a peer-to-peer storage system. It is highly-available and scalable due to it's peer-to-peer nature. It's often compared to Honeycomb, but there are some substantial differences. Honeycomb was targeted specifically at fixed-content storage. It's reliability was built around the use of the Reed-Solomon encoding. And there was a close coupling of data and metadata in Honeycomb (providing the same failure characteristics for both).

Celeste does split an 'object' into blocks which get dispersed into the Celeste storage 'cloud'. It is possible to specify the number of copies of each object that is stored in the system. Celeste is also an application on top of the Beehive framework, but I need to spend a bunch more time looking into that before I understand how these interrelate.

There's a good article talking to Glenn Scott about Celeste which goes into a lot more details than I've mentioned above.

Monday Nov 24, 2008

Why am I so stupid?

I always tend to prefer stuff on the bleeding edge warts and all. So I was in Baltimore for PASIG during the week and I suggested to a few folks they should consider using Celeste and a basis for a data store, seems fine right? It worked out of the box for me on OpenSolaris, they had a few problems so I rebooted into Ubuntu to install and try it there. Again no problems.

So whilst in Ubuntu I decided to use zfs-fuse to pull in my mail filters for Thunderbird. Again should be simple enough. I'd had problems before with this causing problems to the underlying zpool, but I thought this would be fine since I just wanted read access. Big mistake.

Ok, so what happens? When I try to reboot the ZFS filesystem isn't recognised by GRUB. Ok that's fine, I've seen it before. Simple solution, boot with the LiveCD import the pool done, right? Nope, unfortunately no LiveCD and I'm also not patient (hence the title of this entry). So what do I do next? Try and fail to create a OpenSolaris Live USB keyring. Then I get the idea that I should update ZFS fuse and see if there is anything I can do.

hg update, scons, sudo scons install. Good to go. Then I notice the ZFS version on the pool is only 10 and ZFS fuse goes all the way to 14. Maybe that's the problem.. This is where the good fairy should have been crying "Don't do it, Don't do it". Unfortunately there was no one around to pry my fingers away from the keyboard and I upgraded the filesystem.

Now ZFS doesn't see the pool at all... Now that I'm back in the office I'll try to do some surgery with zdb, but I'm not hopeful..

About

Peter Buckingham

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today