Binary shortlist loading from bytebuffer

[browsermt]

Binary shortlist loading from bytebuffer

[kpu]

Stop creating more of a mess. Get Nick's PR in.

[qianqianzhu]

@jerinphilip Can you first get browsermt/marian-dev#26 merged? Then I can have the new BinaryShortlistGenerator to make changes.

[kpu]

Merge into browsermt/marian-dev#26 done and Nick's PR is in. I've demanded @jerinphilip simplify Service (#61) to decrease the tax imposed by too many classes.

[jerinphilip]

Hello @qianqianzhu. Your scope ends at demonstrating functionality at the boundary of Service, which is now collapsed into a single class with ifdefs.

Probably keep a watch on how #63 moves, as you'll need similar datatypes for shortlist as well?

Service handles interaction of translation components.

bergamot-translator/src/translator/service.h

Line 43 in 34228d3

explicit Service(Ptr<Options> options, const void *model_memory = nullptr);

.

You will have to get the shortlist memory down to batchtranslator through:

bergamot-translator/src/translator/service.cpp

Line 57 in 34228d3

translators_.emplace_back(deviceId, vocabs_, options, model_memory_);

bergamot-translator/src/translator/batch_translator.h

Line 36 in 34228d3

explicit BatchTranslator(DeviceId const device, std::vector<Ptr<Vocab const>> &vocabs,

BatchTranslator will look like translator or server parts which you already have edited to bring the changes to marian-dev.

To test functionality, please follow @XapaJIaMnu on #55, but do not edit the unified API boundaries (TranslationModel, AbstractTranslationModel) as it can potentially lead to WASM build errors etc. You might just want to incorporate the bytearray shortlist test just below:

bergamot-translator/app/service-cli-bytearray.cpp

Line 18 in 34228d3

void * model_bytes = bergamot::getBinaryModelFromConfig(options);

[qianqianzhu]

Just made a new PR (browsermt/marian-dev#28 (comment)) to enable binary shortlist loading from bytebuffer (which was missed last time). Also, if this PR get merged, can we update marian-dev submodule?

[jerinphilip]

@qianqianzhu Looks like kpu seems to be squash merging small features.

can we update marian-dev submodule?

This needn't block your feature development. Get the functionality (in bergamot-translator) ready within your PR by change of submodule (you can change the submodule as you want in shortlist_plumb branch) and bring the change in submodule in with your shortlist PR here. Given the squash merges, multiple jumps aren't likely to pollute history.

(There's a mess in marian-dev submodules due to some wasm divergence, ideally don't bother, just develop your functionality, nothing till Service should be affected).

[qianqianzhu]

Thanks! If this is fine to make changes, I will make a PR soon.

[abhi-agg]

@qianqianzhu Can we close this issue or is there anything else that is left to be done before closing it?

[kpu]

@abhi-agg From our perspective it's ready for javascript to give us the bytes.