qzsl123/minidfs

Simple version of Distributed File System on multiple threads

miniDFS

Simple version of Distributed File System

Design a Mini Distributed File System (Mini-DFS), which contains

• A client
• A name server
• Four data servers

Mini-DFS is running through a process. In this process, the name server and data servers are different threads

Basic functions of Mini-DFS

• Read/write a file
• Upload a file: upload success and return the ID of the file
• Read the location of a file based on the file ID and the offset

File striping

• Slicing a file into several chunks
• Each chunk is 2MB
• Uniform distribution of these chunks among four data servers

Replication

• Each chunk has three replications
• Replicas are distributed in different data servers

Name Server

• List the relationships between file and chunks
• List the relationships between replicas and data servers
• Data server management

Data Server

• Read/Write a local chunk
• Write a chunk via a local directory path

Client

• Provide read/write interfaces of a file

Mini-DFS can show

• Read a file (more than 7MB)
• Via input the file and directory
• Write a file (more than 3MB)
• Each data server should contain appropriate number of chunks
• Using MD5 checksum for a chunk in different data servers, the results should be the same
• Check a file in (or not in) Mini-DFS via inputting a given directory
• By inputting a file and a random offset, output the content

compile

make

run

./miniDFS
put source_file_path dest_file_path
ls
locate FileID Offset
fetch FileID Offset dest_file_path
...
read source_file_path dest_file_path
...