gan-tu/cppgfs2.0

A distributed Google File System (GFS), partially implemented in C++.

cppgfs2.0's Issues

Metadata Manager

1. handles how we store and read the file metadata: (filename, chunk_index) -> chunk locations
2. handles creation of new metadata for file creation and snapshot operations.
3. when a chunk server starts/restarts and connects to master, the metadata manager is responsible for updating its metadata to include the chunks this chunk server has in its "locations" mapping

Build error

It showed the below error when trying to build in Mac OS X.

ERROR: /private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/com_google_protobuf/BUILD:975:21: in proto_lang_toolchain rule @com_google_protobuf//:cc_toolchain: '@com_google_protobuf//:cc_toolchain' does not have mandatory provider 'ProtoInfo'.
INFO: Repository boringssl instantiated at:
/Users/balachandhar/Documents/cppGFS2.0/WORKSPACE:52:10: in
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/com_github_grpc_grpc/bazel/grpc_deps.bzl:130:21: in grpc_deps
Repository rule http_archive defined at:
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in
INFO: Repository remotejdk11_macos instantiated at:
/DEFAULT.WORKSPACE.SUFFIX:102:6: in
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/bazel_tools/tools/build_defs/repo/utils.bzl:201:18: in maybe
Repository rule http_archive defined at:
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in
INFO: Repository remote_java_tools_darwin instantiated at:
/DEFAULT.WORKSPACE.SUFFIX:219:6: in
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/bazel_tools/tools/build_defs/repo/utils.bzl:201:18: in maybe
Repository rule http_archive defined at:
/private/var/tmp/_bazel_balachandhar/14943156afbb9d37410525ecd02ad6fd/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in
ERROR: Analysis of target '//src/protos/grpc:cc_chunk_server_lease_service_grpc' failed; build aborted: Analysis of target '@com_google_protobuf//:cc_toolchain' failed
INFO: Elapsed time: 8.781s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (72 packages loaded, 2959 targets configured)

System Log Manager

Handles writing system logs. Can be to console/file. Can consider Log4cxx if trivial.

Bazel's rules_foreign_cc may be useful here

Master file namespace Lock manager

Acquires proper read/write locks on file namespaces for concurrent master operations to properly serialize its operational order

Update: the lock is released when replying to client, and this is NOT holding read/write locks for the client, but simply for master's own concurrency/serialization purposes

File Chunk Manager

Handles storing and retrieving chunks.

Can use LevelDB for this; for BUILD, consider using rule_cc_foreign or this pull request

Communication Manager

Uses gRPC to handle communications between nodes.

This defines all the gRPC endpoints that client, master, chunk server has to individually implement, and handles the bulk of gRPC communication work.

Note that this means after this module is finished, implementations for each grpc call are done individually by client, master, and chunk server code. This is where our crucial business logics are.

Chunk Server Manager

1. Knows who and where each chunk is, using communication manager
2. Decides which chunk server to place new file chunks, for load balancing effects
3. Maintains a Heartbeat Manager for tracking chunk server health status, and asks re-replication manager to bring replica counts up to minimum replica count, when required (this is phase 2/3)

Write Lease Manager

1. Handles lease grant/revoke/renew for primary replica used for write operations.
2. Appoints primary replica and increment chunk version number for all chunk servers holding that chunk, when necessary.

See below for high level explanation:

Note: I found this has high overlap between metadata manager and chunk server manager, and michael@ may also wants more things to do, so I instead assign this to both of you to coordinate

Configuration Manager

Implements how nodes read configs. Such as IPs, directories etc. can either read from yaml/toml file, command line, or environment variables defined by Dockerfile

GFS Client API

Public client API exposed to end users for supported GFS file operations. This is what a user of our GFS uses to access files. This is also what we use for integration testing.

This will be open, close, create, read, and write APIs. Every one, except close will talk to a server with GFS logics; close is simply deallocating memory resources used by client for the given file operation session