MooseFS is a distributed file system that aims to be a fault-tolerant, highly available and performing, scalable general-purpose network distributed file system for data centers. In this article I’ll show how I’ve started working with MooseFS.
To start working with MooseFS you’ll need a couple of things:
- Several servers, as this is a distributed file system:
- A master server – it can be more than one with MooseFS Pro – for managing the filesystem and storing metadata.
- Metalogger backup server, to store metadata changelogs and can be used to set up as master in case of failures. Any number of them.
- Chunk (data) servers, to store file data and synchronize among them. The more, the better. You can keep several copies of the same data to improve reliability.
- Clients: someone has to work with the stored files, right? 😉
- A running DNS system. Here in unixcop.com, we like bind (every word leads to a different bind tutorial)
- The MooseFS user’s manual. There is also an step-by-step guide… but this article intends to replace that guide
I don’t have at home so many computers, but I can use gnome-boxes to create several Virtual Machines. Four in total: master server, metalogger and two chunk servers. All of them with Debian, just because. Also, the master will serve the DNS service.
Add moosefs repository
This is a common step for every server, no matter what role. My recomendation, specially if you are going to use VMs is to install this first, then copy this VM to create the other servers.
Install Debian as usual, and then run as root:
root@vm: # wget -O - http://ppa.moosefs.com/moosefs.key | apt-key add - root@vm: # echo "deb [arch=amd64] http://ppa.moosefs.com/moosefs-3/apt/debian/bullseye bullseye main" > /etc/apt/sources.list.d/moosefs.list root@vm: # apt-get update
Now we are ready to clone our VM as needed and proceed with the Master server installation.
Launch the virtual machine for the master server and run:
root@vm: # apt-get install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli
Then you can start the services with:
# systemctl start moosefs-master # systemctl start moosefs-cgiserv
Actually, the CGI server isn’t mandatory, but you’ll get a nice status monitor that looks like this:
After the CGI server started, go to http://your.mfs.master.server:9425/ to see it in action. Don’t forget to update your DNS with the newly created VMs.
If you don’t want to run the cgi server, you can run the CLI monitor:
# mfscli -f1 -SIN -H your.mfs.master.server
where -f1 is to use thick unicode frames. -SIN means show full master info; and -H is the master server to connect.
On the metalogger server run:
root@meta # apt-get install moosefs-metalogger
Before we start the service, we need to create a configuration file and set some variables:
root@meta: # cd /etc/mfs root@meta: # cp mfsmetalogger.cfg.sample mfsmetalogger.cfg root@meta: # nano mfsmetalogger.cfg #or your favorite text editor
And look for MASTER_HOST and set to the hostname of your master server.
Now we can start the metalogger service:
root@meta: # systemctl start moosefs-metalogger
Prepare the disk
The chunk servers should have (at least) an extra disk to store the data. We will partition and format with XFS. You can use any tool you want (i.e. gparted), or do it as I did on the following screenshot sequence:
The first step is to install the chunkserver service:
root@chunk1: # apt-get install moosefs-chunkserver
Before we can start the chunkserver service we need to create the config files:
root@chunk1: # cd /etc/mfs root@chunk1: # cp mfschunkserver.cfg.sample mfschunkserver.cfg root@chunk1: # cp mfshdd.cfg.sample mfshdd.cfg
The file mfshdd.cfg is pretty simple, just define the paths for the drives to export with the chunk server. /mnt/chunks1 according to my fstab file.
And on mfschunkserver.cfg we set the master host location in the MASTER_HOST variable:
Now we can start the chunkserver service
root@chunk1: # systemctl start moosefs-chunkserver
If you go with your browser to the status monitor you’ll se some space available, yay!
Now repeat those steps with the other(s) server(s) and observe the results:
Click on the Servers + tab to see information about our chunkservers:
It’s nice to have storage space available, but is nicer to use that space, right? To start working with MooseFS I need a fuse client that is installed with:
root@client: # apt-get install fuse libfuse2 moosefs-client
Then you can mount the moosefs filesystem with mfsmount -H <mfs.master.server> <mountpoint>, like this:
On the screenshot I’ve also checked the space available (df -h) and created some random files with random content. And of course, I’ve check again that nice status screen: