sandeepkru/kosmosfs

Automatically exported from code.google.com/p/kosmosfs

kosmosfs's Issues

Complication problems on CentOS 6.3 server

Complication problems occured when Linking CXX executable filelister:
CMakeFiles/filelister.dir/filelister_main.o: In function 
'__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to 
'boost::system:;generic_category()'
CMakeFiles/filelister.dir/filelister_main.o: In function 'thread_exception':
/usr/local/include/boost/thread/exceptions.hpp:51 undefined reference to 
'boost::system:;generic_category()'
libkfsMeta.a(checkpoint.o):In function 
'__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to 
'boost::system:;generic_category()'
libkfsMeta.a(kfsops.o):In function '__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to 
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to 
'boost::system:;generic_category()'
....
....
....
....


collect2: ld 返回1
gmake[2]: *** [src/cc/meta/filelister] 错误 1
gmake[1]: *** [src/cc/meta/CMakeFiles/cpfromkfs.dir/all] 错误 2
gmake: *** [all] 错误 2

Original issue reported on code.google.com by [email protected] on 25 May 2013 at 9:31

Error while mounting kfs directory using FUSE

Error while mounting KFS directory  using FUSE ? 

./kfs_fuse /media/hugedisk/KFS/mountkfs -f

kfs_fuse: /usr/include/boost/smart_ptr/shared_ptr.hpp:418: T* 
boost::shared_ptr< <template-parameter-1-1> >::operator->() const [with T = 
KFS::KfsClient]: Assertion `px != 0' failed.
Aborted


What version of the product are you using? On what operating system?
Im using kosmosfs .5 version on Ubuntu 10.04 64 bit .. 

Original issue reported on code.google.com by [email protected] on 10 Jun 2011 at 12:33

Pure documentation. README is old. Binaries from build/bin don't render "description" when launched with -h or --help.

Great project first of all! Respect to you kosmos ppl. 
But any chance that we(users) can have more broad and comprehensible 
documenation on binaries and scripts?

For example:
$ ./telemetry_server --help
./telemetry_server: invalid option -- '-'
./telemetry_server: invalid option -- 'e'
Usage: ./telemetry_server -m <machines file>  -l <log fn>

Program tells me what usage should I use, but it doesn't help user to 
understood, actually, what is "telemetry_server" by itself  :)

This kind of problem affects all binaries - lack of description. 

Or this one:
$ ./rebalancer --help
Usage: ./rebalancer <network defn> <data dir> <outputFn>

Any chance to know what is <network defn> ??


Thank you in advance. 
In anycase, great job!

Original issue reported on code.google.com by [email protected] on 7 Jun 2012 at 10:50

kfs-0.5 setup problem

operation system: centos 6.2 64-bit

kfs version: kfs-0.5 


[root@gifer scripts]# python kfssetup.py -f machines.cfg  -b ../build -w 
../webui -s
kfssetup.py:28: DeprecationWarning: The popen2 module is deprecated.  Use the 
subprocess module.
  import socket,threading,popen2
kfssetup.py:29: DeprecationWarning: the md5 module is deprecated; use hashlib 
instead
  import md5
cp: 略过目录"./scripts"
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: metaserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping

[root@gifer tools]#  ./kfsping -c -s localhost -p 20000
Meta-server:  0
Total-space: 0 (MB)
Used-space: 0 (MB)

is setup success?

if no what should i do ?

thank you!

Original issue reported on code.google.com by [email protected] on 21 May 2012 at 8:38

TryToStart Invalid argument 22Aborted

KFS 0.5/Debian squeeze/mipsel/2.6.32-5-4kc-malta/PAGESIZE=4096

metaserver and kfsping exited with the error message as follows:
TryToStart Invalid argument 22Aborted

In src/cc/qcdio/qcthread.cpp:106
if (inStackSize > 0 && (theErr = pthread_attr_setstacksize(
            &theStackSizeAttr, inStackSize)) != 0)

pthread_attr_setstacksize failed when inStackSize is 32KB. I wrote a small test 
program:
    pthread_attr_getstacksize(&a, &n);
    printf("%d\n", n);
    for (n = 16384; n <= 256 * 1024; n *= 2)
        printf("%d\t%d\n", n, pthread_attr_setstacksize(&a, n));

On x86:
8388608
16384   0
32768   0
65536   0
131072  0
262144  0

On the machine:
8388608
16384   22
32768   22
65536   22
131072  0
262144  0

I knew that's not KFS' fault, but 128KB seems to be more adaptable. 

`man pthread_attr_setstacksize`:
pthread_attr_setstacksize() can fail with the following error:
EINVAL The stack size is less than PTHREAD_STACK_MIN (16384) bytes.

On some systems, pthread_attr_setstacksize() can fail with the error EINVAL if 
stacksize is not a multiple of the system page size.

Original issue reported on code.google.com by [email protected] on 25 Jul 2012 at 10:45

kfs 0.5 fuse bug

What steps will reproduce the problem?
1. kfs_fuse mountpoint
2. try to cp a small file 

Gives error on cp: Numerical result out of range

ChunkServer: 
12-12-2011 02:45:00.469 DEBUG - (MetaServerSM.cc:411) recv meta reply: seq: 
2516678953956645717 status: 0 corrupt chunk:  fileid = 8 chunkid = 139
12-12-2011 02:45:06.980 DEBUG - (MetaServerSM.cc:479) recv meta cmd: seq: 
335567769309155733 alloc-chunk: seq: 335567769309155733 fileid: 8 chunkid: 140 
chunkvers: 1 leaseid: -1 append: 0

Can't make it to work even on basic files like an image. 


Ubuntu 11.04, kfs 0.5

Original issue reported on code.google.com by [email protected] on 12 Dec 2011 at 12:55

Problems during gmake on Centos6 64bit

What steps will reproduce the problem?
1. After doing gmake when I'm building C++ code

What is the expected output? What do you see instead?
Many errors during gmake commmand. When files treat the contents of a 
xfs/linux.h (with an include command) some function in this file don't work 
correctly (cannot convert, was not declared in this scope).  

What version of the product are you using? On what operating system?
I'm trying to build kfs-0.5 on Centos6.3 64bit with xfsprogs.x86_64 and 
xfsprogs-devel.x86_64 packs

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 12 Jul 2012 at 1:56

Attachments:

• error.png

No File Permission Support?

I tried to use KFS-0.5 with Hadoop-1.0.3. When I ran a job,
I got the error information:
ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop 
cause:java.io.IOException: The ownership/permissions on the staging directory 
kfs://mars:20000/tmp/hadoop-hadoop/mapred/staging/hadoop/.staging is not as 
expected. It is owned by  and permissions are rwxrwxrwx. The directory must be 
owned by the submitter hadoop or by hadoop and permissions must be rwx------

`bin/hadoop fs -ls` shows all files have the rwxrwxrwx permission.
kfsshell doesn't list permission.
class KfsClient doesn't seem to have something like chmod or chown.

Does KFS not support file permission?

Thanks,
Gary

Original issue reported on code.google.com by [email protected] on 31 Jul 2012 at 1:49

why error, when do Read&Write in open mode O_RDWR

What steps will reproduce the problem?
1. open file in O_RDWR
2. Write file
3. Read file (error)

What is the expected output? What do you see instead?
 INFO - (KfsClient.cc:3317) Server says lease on chunk ** is busy...waiting

What version of the product are you using? On what operating system?
 KFS-0.5, linux (ubuntu）

Please provide any additional information below.

why will this error happen?
how to deal with it? see the attachment, modified from the kfsSample.cpp

thanks vary much...

Original issue reported on code.google.com by [email protected] on 14 Aug 2012 at 10:21

Attachments:

• kfsSample1.cpp

data structure in KFS to locate the chunks in distrubuted nodes

I am initiating a new project which is likely fit into KFS. We have a huge 
number of distributed chunk servers. I am wondering how KFS to locate a chunk. 
Does it have an interface to memcache or database? Please let me know a little 
bit more details. Thank you.

Original issue reported on code.google.com by [email protected] on 4 Dec 2012 at 2:22

bug in WriteAppender.cc

bug in file 
kfs-0.5/src/cc/libkfsClient/WriteAppender.cc
http://code.google.com/p/kosmosfs/source/browse/trunk/src/cc/libkfsClient/WriteA
ppender.cc

void SpaceRelease()
{
    if (mSpaceAvailable <= 0) {
            StartAppend();
        return;
    }
        Reset(mSpaceReleaseOp);
        mSpaceReleaseOp.chunkId      = mAllocOp.chunkId;
//! bug: should mAllocOp.chunkVersion; NOT ,
        mSpaceReleaseOp.chunkVersion = mAllocOp.chunkVersion; //! , --> ;
    mSpaceReleaseOp.writeInfo    = mWriteIds;
    mSpaceReleaseOp.numBytes     = size_t(mSpaceAvailable);
    Enqueue(mSpaceReleaseOp);
}
chenbk

Original issue reported on code.google.com by [email protected] on 30 Nov 2011 at 3:10

I can't use the tools after deploying kfs

After deploying the kfs, i can' use the tools. I tried the kfsping command 
(kfsping -m -s kfs1 -p 20000) and he drops the following message:

-bash: kfsping: command not found

Same with cptokfs...

I'm using KFS 0.5 version on Linux Debian Squeeze 32-bit(kernel version 
2.6.32-5-686)

Please help.

Original issue reported on code.google.com by [email protected] on 5 Apr 2012 at 4:53

Compilation problem on ubuntu 11.10 64bit server

Linking CXX executable cpfromkfs
../qcdio/libqcdio.a(qcmutex.o): In function `GetAbsTimeout':
/home/septem/kfs/kfs-0.5/src/cc/qcdio/qcmutex.cpp:40: undefined reference to 
`clock_gettime'
/home/septem/kfs/kfs-0.5/src/cc/qcdio/qcmutex.cpp:40: undefined reference to 
`clock_gettime'
collect2: ld returned 1 exit status
make[2]: *** [src/cc/tools/cpfromkfs] Error 1
make[1]: *** [src/cc/tools/CMakeFiles/cpfromkfs.dir/all] Error 2
make: *** [all] Error 2

Original issue reported on code.google.com by [email protected] on 29 Dec 2011 at 5:55

bugs

1.there are bugs In Kfs_fuse_main.cc 
2.when client call write(nfd,buf,nwrite) function to write data into dfs,
why the return size is even bigger than nwrite size .Lt is absurd. l print the 
log
and find that it returns the total size that have been written to dfs.
cause l call {int res=write(nfd,buf,4096)} for serval times.
and res is 4096,8192...
l thought maybe it is because of the default buffer in client.
so l call function SetIoBufferSize(nfd,0),but it doesn't work.
what a fun it is .the write function returns some size bigger than 4096,and the 
file is even closed in client,but the file size in dfs is still 4096 big.

so many bugs in the project

Original issue reported on code.google.com by [email protected] on 13 May 2012 at 12:08

INFO - (KfsClient.cc:2043) Unable to get size for chunk 238; error = -9; need to retry

What steps will reproduce the problem?

1. kfs metaserver chunkserver deployed on a machine
2. when I called  the function Write() on another machine this occured.  
3. AllocChunk()failed sizechunk(fd)<0


What version of the product are you using? On what operating system?
kfs-0.5  centos

Please provide any additional information below.

I do not know the reason

Thanks advance!

Original issue reported on code.google.com by [email protected] on 7 Jan 2014 at 10:03

can't use seek() function

operation system: centos 6.2 64-bit

kfs version: kfs-0.5 

when I compile the KfsSample_main.cc file, it has same erros: 


make all 
Building file: ../kfstest.cpp
Invoking: GCC C++ Compiler
g++ -I/home/altair/code/kfs/include/kfs -O0 -g3 -Wall -c -fmessage-length=0 
-MMD -MP -MF"kfstest.d" -MT"kfstest.d" -o"kfstest.o" "../kfstest.cpp"
Finished building: ../kfstest.cpp

Building target: KfsTestSample
Invoking: GCC C++ Linker
g++  -o"KfsTestSample"  ./kfstest.o  
/home/altair/code/kfs/lib/static/libkfsClient.a 
/home/altair/code/kfs/lib/static/libkfsCommon.a 
/home/altair/code/kfs/lib/static/libkfsEmulator.a 
/home/altair/code/kfs/lib/static/libkfsIO.a 
/home/altair/code/kfs/lib/static/libkfsMeta.a 
/home/altair/code/kfs/lib/static/libqcdio.a 
/home/altair/code/kfs/lib/static/libtools.a -lpthread -lcrypto -lrt
./kfstest.o: In function `main':
/home/altair/workspace/KfsTestSample/Debug/../kfstest.cpp:193: undefined 
reference to `KFS::KfsClient::Seek(int, long)'
collect2: ld returned 1 exit status
make: *** [KfsTestSample] Error 1


the code that I use seek() is:
    gKfsClient->Seek(fd, (off_t)40);


the complier force (off_t)40 convert to long int;and can't find 
Seek（int,off_t) define.Has I done something wrong?Hope to get some help,thank!

Original issue reported on code.google.com by [email protected] on 29 Jun 2012 at 6:16

Meta Server and Chunk Server can't start, Why?

after, command "python kfssetup.py -f machines.cfg -m machines.txt -b ../build 
-w ../webui" and command "python kfslaunch.py -f machines.cfg -m machines.txt 
-s", I tend to run command "kfsping -m -s 192.168.1.107 -p 20000", but can not 
connect.

It display the following info

[root@bogon tools]# ./kfsping -m -s 192.168.1.107 -p 20000
Connect: : Connection refused
02-19-2012 14:23:40.960 ERROR - (kfsping_main.cc:127) Unable to connect to 
192.168.1.107 20000

actually, Meta Server don't run.


So, I use cmd "./metaserver MetaServer.prp", shell tell me that

Loading key metaServer.clientPort with value 20000
Loading key metaServer.chunkServerPort with value 30000
Loading key metaServer.logDir with value ./kfslog
Loading key metaServer.cpDir with value ./kfscp
Using meta server client port: 20000
Using meta server chunk server port: 30000
Setting cluster key to:
Setting md5sumFilename to:
min. # of chunkserver that should connect: 1
min. # of replicas per file: 1
max. # of replicas per file: 64
Setting the # of open files to: 1024
restoring from checkpoint of Tue Feb 14 23:06:31 2012
log file is ./kfslog/log.0
Log time: Tue Feb 14 23:06:31 2012
kfs_startup() end
02-20-2012 16:39:45.170 INFO - (metaserver_main.cc:88) Starting metaserver...
02-20-2012 16:39:45.203 INFO - (replay.cc:57) Doing log replay from file: 
./kfslog/log.0
02-20-2012 16:39:45.204 INFO - (replay.cc:57) Doing log replay from file: 
./kfslog/log.0
02-20-2012 16:39:45.231 DEBUG - (logger.cc:122) Opening ./kfslog/log.0 in 
append mode


and no response.


why I can't start meta server and chunk server??



OS: Centos 5.7 x86_64
KFS Code: kfs-0.5



Thank you very much!

Original issue reported on code.google.com by [email protected] on 20 Feb 2012 at 8:48

Error when linking client test program

I finished compiling and deploying KFS. And made the servers start. I want to 
have a test on client programming. But I get link errors like these:

/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::re_detail::cpp_regex_traits_implementation<char>::transform_primary(char
 const*, char const*) const'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::re_detail::get_default_error_string(boost::regex_constants::error_type)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::re_detail::cpp_regex_traits_implementation<char>::transform(char 
const*, char const*) const'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::re_detail::put_mem_block(void*)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::basic_regex<char, boost::regex_traits<char, 
boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*, unsigned 
int)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to 
`boost::re_detail::raise_runtime_error(std::runtime_error const&)'

I give a snapshot of my code and make output 
[here](https://gist.github.com/ggicci/10342928).

Original issue reported on code.google.com by [email protected] on 10 Apr 2014 at 12:32

Are there any design documents

Are there any design documents for the purpose of code analyzing
such as uml graph,anything will be fine

Original issue reported on code.google.com by [email protected] on 11 Aug 2011 at 10:54