[scores] хватит смотреть в один bleu score, надо что-нибудь визуализировать, чтобы лучше понимать, что происходит

[natalymr]

[scores] хватит смотреть в один bleu score, надо что-нибудь визуализировать, чтобы лучше понимать, что происходит

[natalymr]

Посмотрела какие-то промежуточные результаты на camel-e, BLEU = 6.01, "большой" val

Так как гитхабовские сервера не справляются рендерить ноутбуки, то вставляю картинкой график здесь

Так же вывела все сообщения, у которых bleu-score больше 0, в порядке увеличения скора:

BLEU: 7.91
Reference: camel <num> camel aws s <num> add a parameter to specify chunked encoding disabled option change name to the option regen docs.
Prediction: camel <num> camel aws since we are using builders we

BLEU: 13.04
Reference: camel <num> camel aws s <num> add parameters to specify s <num> client options accelerate mode option.
Prediction: camel <num> camel aws create single spring boot itests for

BLEU: 13.04
Reference: camel <num> camel aws kinesis firehose add the ability to specify credentials and region at component level.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 13.07
Reference: camel <num> camel aws lets adopt the same approach as camel <num> for all the components camel aws lambda.
Prediction: camel aws lets adopt the value of fixed

BLEU: 13.53
Reference: camel <num> shutdown only aws clients owned by the context aws mq.
Prediction: shutdown only aws clients

BLEU: 13.79
Reference: camel <num> camel aws kinesis firehose use a configuration for the options like the other aws components.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 14.42
Reference: camel <num> camel aws mq add the ability to specify credentials and region at component level.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 14.69
Reference: camel <num> camel aws lets adopt the same approach as camel <num> for all the components camel aws cw.
Prediction: camel aws lets adopt the value of different only

BLEU: 17.61
Reference: camel <num> camel ftp tries reconnects twice as much as maximum reconnect attempts <num>.
Prediction: camel <num> camel ftp spi option to turn different propgate

BLEU: 19.14
Reference: camel <num> camel jms suspending resuming won t work in case of timeout.
Prediction: camel <num> camel jms add spring boot for jms

BLEU: 19.46
Reference: camel <num> service call improve configuration of service call components in java dsl.
Prediction: camel <num> service call add an option to configure a

BLEU: 20.42
Reference: camel <num> camel aws add component verifiers like s <num> for all the aws components lambda component.
Prediction: camel <num> camel aws add an option to specify the

BLEU: 21.51
Reference: camel <num> camel aws sns make the autocreation of the topic configurable.
Prediction: camel <num> camel aws since we are using builders we

BLEU: 21.51
Reference: camel <num> camel aws swf region is not set during client creation.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 22.44
Reference: camel <num> camel aws use regional client instead of define region for single operation aws ec <num>.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 23.67
Reference: camel <num> camel <num> use bind to registry wherever possible camel websocket.
Prediction: camel irc use bind to registry annotation where possible

BLEU: 23.9
Reference: camel <num> optimise bean processor make light weight not as service.
Prediction: camel <num> optimise bean expression bean to bean start

BLEU: 24.12
Reference: camel <num> camel <num> use bind to registry wherever possible camel xmpp.
Prediction: camel <num> camel <num> upgrade to use registry

BLEU: 24.8
Reference: camel <num> camel aws use regional client instead of define region for single operation aws sdb.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 25.13
Reference: camel <num> camel main added option to log summary or not.
Prediction: camel <num> camel main allow to configure global and common

BLEU: 25.27
Reference: camel <num> camel jmx allow consumer to alos filter observed attribute.
Prediction: camel <num> camel jmx add missing data to specify

BLEU: 26.27
Reference: camel <num> camel main output summary of auto configuration.
Prediction: camel <num> camel main maven plugin to generte spring boot

BLEU: 26.27
Reference: camel <num> camel hazelcast support reliable topic.
Prediction: camel <num> camel hazelcast allow to configure a list of

BLEU: 26.3
Reference: camel <num> add an advanced properties to the camel thrift component.
Prediction: camel <num> add an option to configure component

BLEU: 26.71
Reference: camel <num> camel connector allow to configure sb config prefix.
Prediction: camel <num> camel connector generate spring boot auto configuration

BLEU: 27.78
Reference: camel <num> camel aws create an iam component first commit.
Prediction: camel <num> camel aws iam add the ability to specify

BLEU: 27.78
Reference: camel <num> camel aws ddbstream add a component verifier.
Prediction: camel <num> camel aws sqs add the ability to specify

BLEU: 28.62
Reference: camel aws sqs add a consumer integration test for delete if filtered option.
Prediction: camel <num> camel aws sqs add a list polished

BLEU: 29.85
Reference: camel <num> camel ftp more unittest on osgi parsers.
Prediction: camel <num> camel ftp add option for resume download

BLEU: 29.95
Reference: camel <num> camel <num> use bind to registry wherever possible camel sjms <num>.
Prediction: camel <num> camel <num> use kafka camel instead of

BLEU: 30.86
Reference: camel <num> camel aws add component verifiers like s <num> for all the aws components sns component.
Prediction: camel <num> camel aws add component verifiers component verifier

BLEU: 32.47
Reference: camel hipchat use bind to registry annotation wherever possible.
Prediction: camel <num> camel <num> use bind to registry wherever possible

BLEU: 33.03
Reference: camel crypto use bind to registry annotation wherever possible.
Prediction: camel <num> add bind to registry annotation where possible

BLEU: 35.54
Reference: camel <num> rest dsl producer should use upper case http verb.
Prediction: camel <num> rest dsl producer issue with npe or other

BLEU: 38.3
Reference: camel <num> camel aws lambda add support for event source mapping delete operation.
Prediction: camel <num> camel aws lambda add the ability to specify

BLEU: 40.66
Reference: camel <num> camel aws lets adopt the same approach as camel <num> for all the components camel aws sns.
Prediction: camel <num> camel aws lets adopt the same approach as

BLEU: 40.97
Reference: camel <num> camel ftp add option to turn on logging of transfer activity.
Prediction: camel <num> camel ftp add option and option to

BLEU: 41.11
Reference: camel <num> camel elasticsearch rest remove info operation from producer.
Prediction: camel <num> camel elasticsearch rest producer should not support for

BLEU: 42.73
Reference: camel metrics use bind to registry annotation wherever possible.
Prediction: camel aws mq use bind to registry annotation where possible

BLEU: 43.74
Reference: camel <num> camel <num> use bind to registry wherever possible camel stream.
Prediction: camel <num> use bind to registry annotation where possible

BLEU: 44.93
Reference: camel <num> camel aws add component verifiers like s <num> for all the aws components ec <num> component.
Prediction: camel <num> camel aws add component verifiers like s <num>

BLEU: 48.75
Reference: camel <num> upgrade mockito core to version <num> <num> for camel spring redis.
Prediction: camel <num> upgrade mockito core to <num> <num> for polish

BLEU: 49.66
Reference: camel <num> rename components to default names camel netty <num> to camel netty aligned camel netty tests.
Prediction: camel <num> rename components to default names camel netty <num>

BLEU: 49.66
Reference: camel <num> rename components to default names camel netty <num> to camel netty fixed camel ganglia test.
Prediction: camel <num> rename components to default names camel netty <num>

BLEU: 49.66
Reference: camel <num> rename components to default names camel quartz <num> to camel quartz fixed references in javadoc.
Prediction: camel <num> rename components to default names camel quartz <num>

BLEU: 49.66
Reference: camel <num> camel aws add component verifiers like s <num> for all the aws components ses component.
Prediction: camel <num> camel aws add component verifiers like s <num>

BLEU: 53.73
Reference: camel google pubsub use bind to registry wherever possible.
Prediction: camel <num> camel <num> use bind to registry wherever possible

BLEU: 54.88
Reference: camel <num> add unit test for zero timeout.
Prediction: camel <num> add unit test

BLEU: 57.89
Reference: camel aws msk use bind to registry annotation where possible in tests.
Prediction: camel aws mq use bind to registry annotation where possible

BLEU: 57.97
Reference: camel <num> rename components to default names camel mina <num> to camel mina.
Prediction: camel <num> rename components to default names camel quartz <num>

BLEU: 60.65
Reference: camel <num> rename components to default names camel netty <num> to camel netty updated references.
Prediction: camel <num> rename components to default names camel netty <num>

BLEU: 60.65
Reference: camel <num> fixed cs and tests.
Prediction: camel <num> fixed cs

BLEU: 67.03
Reference: camel <num> camel <num> use bind to registry wherever possible camel netty <num> http.
Prediction: camel <num> camel <num> use bind to registry wherever possible

BLEU: 70.71
Reference: camel aws sqs use regions value of in the verifier.
Prediction: camel aws ddb use regions value of in the verifier

BLEU: 70.71
Reference: camel aws mq use regions value of in the verifier.
Prediction: camel aws ddb use regions value of in the verifier

BLEU: 74.08
Reference: camel <num> rename components to default names camel netty <num> to camel netty.
Prediction: camel <num> rename components to default names camel netty <num>

BLEU: 74.08
Reference: camel aws create single karaf feature tests for single components aws mq test.
Prediction: camel aws create single karaf feature tests for single components

BLEU: 74.08
Reference: camel <num> camel aws use builders instead of different constructors aws simple db.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 74.08
Reference: camel <num> camel aws use builders instead of different constructors aws dynamo db.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 74.08
Reference: camel <num> camel aws use builders instead of different constructors aws cloud watch.
Prediction: camel <num> camel aws use builders instead of different constructors

BLEU: 75.06
Reference: camel kafka use bind to registry annotation wherever possible.
Prediction: camel kafka use bind to registry annotation where possible

BLEU: 75.06
Reference: camel cometd use bind to registry annotation where possible.
Prediction: camel irc use bind to registry annotation where possible

BLEU: 81.87
Reference: camel <num> camel <num> use bind to registry wherever possible camel zipkin.
Prediction: camel <num> camel <num> use bind to registry wherever possible

BLEU: 81.87
Reference: camel <num> camel <num> use bind to registry wherever possible camel syslog.
Prediction: camel <num> camel <num> use bind to registry wherever possible

[natalymr]

Вот тут статья оценивает качество перевода
https://arxiv.org/pdf/1907.08710.pdf