openrq-team/openrq

openrq's Issues

Symbol size in "symbol header"

Great library, thanks a lot for making it available to us. =)

Question:
In RaptorQ, the FEC Payload ID (4 bytes) precedes every symbol, and carries the source block number and the encoding symbol ID. I noticed that you also add the symbol size (4 bytes) after that in writeTo(ByteBuffer buffer) of class AbstractEncodingPacket. Any reason to do this? It does not seem to be compliant to the IETF spec (RFC 6330).

How to make it work in C#

I was quite pleased to find you can use IKVM to convert this into a C# .Net dll, as long as you compile with Java <= 1.7

http://www.ikvm.net/userguide/ikvmc.html

P.S. Sorry if this isn't the best place to post this news; I just wanted it out there and in a place that Google/people might notice because it took me time to discover. Maybe this info could be placed in the readme, or on the wiki?

Why abandoned?

Just wondering why you guys decided to stop working on this and if there is a better alternative, or is it that there is some security flaw? RFC-6330 looks fascinating and you did a lot of work. Shame to see it come to an end like that.

using in c/c++

is there a port of this in c/c++ . I want to use this in c++

Add maven compilation

Though ant is wonderful, it'd be nice to have maven support for building and distributing this library.

Where is error correction?

Hi,

First, thank you to all contributors of this impressive package - I appreciate the work put into this (and some of the JUnit tips I've picked up like Parameters =).

Second, apologies for hijacking an 'issue' for what may be my extreme ignorance but this seemed like the only place to contact devs/users. I'm an experienced developer but have very limited experience with error correction (some small Reed-Solomon experiments).

I must be doing something wrong because when I manually corrupt 1 (source data) byte in an encoded byte stream (that includes plenty of Repair packets), the corruption is not repaired after decoding. The (JUnit) method is below and uses a class similar to net.fec.openrq.DataIntegrityCheckTest.java. I have played with all the parameters at the beginning of the test to no avail.

Other questions:

1. With a .6 loss value (and using OpenRQ.minRepairSymbols()), there are almost 8 Repair packets per each Source packet. Is this reasonable? I'm guessing that's an extreme setting that causes unusual outcomes
2. Is there a decoder that automatically absorbs 'extra' Repair packets? I have to perform a hack to throw them away.

Thanks for any insights!

Steve

=========================================

public void corruptByte()
{
    int numSrcBlks = 2;
    int dataLen = 16;
    int symbSize = 4;
    int symbolOverhead = 1;
    double loss = .6;
    fecParams = FECParameters.newParameters(dataLen, symbSize, numSrcBlks);

    // Source is 77, 77, 77, ..
    final byte[] data = new byte[fecParams.dataLengthAsInt()];
    Arrays.fill(data, (byte) 77);

    final ArrayDataEncoder enc = OpenRQ.newEncoder(data, fecParams);

    ByteBuffer bb = ByteBuffer.allocate(5000); // something plenty big

    for (SourceBlockEncoder sbEnc : enc.sourceBlockIterable())
    {
        for (EncodingPacket encodingPacketSource : sbEnc.sourcePacketsIterable())
        {
            encodingPacketSource.writeTo(bb);
        }

        int numRepairSymbols = OpenRQ.minRepairSymbols(dataLen / numSrcBlks, symbolOverhead, loss);
        if (numRepairSymbols > 0)
        {
            for (EncodingPacket encodingPacketRepair : sbEnc.repairPacketsIterable(numRepairSymbols))
            {
                encodingPacketRepair.writeTo(bb);
            }
        }
    }

    bb.flip();

    // Corrupt first source byte
    bb.put(8, (byte) 88);

    /*
     * Decode
     */

    ArrayDataDecoder decoder = OpenRQ.newDecoder(fecParams, symbolOverhead);

    /*
     * In order to throw away "extra" repair packets (e.g., after sourceBlockDecoder.isSourceBlockDecoded() == true),
     * extra logic is needed to "peek" until we see next source block's first source packet
     */
    SourceBlockDecoder latestBlockDecoder = decoder.sourceBlock(0);
    Parsed<EncodingPacket> latestParse = decoder.parsePacket(bb, false);

    for (int sbn = 0; sbn < numSrcBlks; sbn++)
    {

        int packNum = 0;
        boolean allBlockPacketsRead = false;
        while (allBlockPacketsRead == false && latestParse.isValid() == true)
        {
            latestBlockDecoder.putEncodingPacket(latestParse.value());
            System.out.println("SB # " + sbn + " packet#=" + packNum + " decoded = "
                    + latestBlockDecoder.isSourceBlockDecoded() + " type=" + latestParse.value().symbolType());
            packNum++;

            latestParse = decoder.parsePacket(bb, false);

            /*
             * Check if we've finished reading packets for this block
             */
            if (latestParse.isValid() == false || (latestBlockDecoder.isSourceBlockDecoded() == true
                    && latestParse.value().symbolType() == SymbolType.SOURCE))
            {
                allBlockPacketsRead = true;
            }
        }

        if (sbn < numSrcBlks - 1)
        {
            latestBlockDecoder = decoder.sourceBlock(sbn + 1);
        }
    }

    byte[] dataArray = decoder.dataArray();
    
    // The following fails - 88 is not "corrected"
    assertArrayEquals(data, dataArray);

}

`

How to use OpenRQ

I'm download the openrq-3.3.2.jar ,but I don't know how to use it?

Could somebody help me? Thanks!

I have setting the path!

In the cmd:

java -cp.openrq-3.3.2.jar

you can see the problem:

Unrecognized option: -cp.openrq-3.3.2.jar
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

phase5 bug: missing nonZeros

Hi, I'm working on a C++ implementation of RaptorQ and I am currently checking for compatibility with your implementation (version 3.3.2)

I managed to fix a couple of bugs in my implementation by comparing the matrices during the various phases, but I might have found a bug in your phase5.

Basically it is as if your nonZeroRowIterator does not work, and sometimes skips a column that is not zero.
I really have not looked much into your code so I can't tell you much more than that...

I checked this by adding a small loop in the for of pidPhase5 in the openrq/LinearSystem.java file. It checks the contents of the row, column by column, and then I compared that to the indexes of the nonZeroRowIterator in the while loop

Examples

A small real worldish example in /test and a "decoding" wiki page would be massively helpful.

RandomAccessDataEncoder/RandomAccessDataDecoder

It would be nice if there was a RandomAccess Encoder/Decoder that took a RandomAccessFile. This would allow for easily encoding massive files from files larger than available memory.