fibber.metrics.metric_base module

class fibber.metrics.metric_base.MetricBase(field, bs=32, **kwargs)

Bases: abc.ABC

Base class for Metrics.

All metrics should be derived from this class.

To implement a new metric, you should at least overwrite the measure_example method.

The simplest metric can be directly computed from a pair of text, in this case, the metric can use the origin and paraphrase args directly.

Other metrics need more information from the data record. For example, text0, text1, or label. Thus the data_record and field are also provided as args.

Some metrics may run more efficiently on a batch of data. In this case, you should overwrite the measure_batch function. If you don’t overwrite batch_call, it will compute the metric of paraphrase_list one by one.

measure_batch(origin, paraphrase_list, data_record=None, **kwargs)

Measure the metric on a batch of paraphrase_list.

If batch is larger than self._bs, the data will be split into smaller batches.

Parameters

• origin (str) – the original text.

• paraphrase_list (list) – a set of paraphrase_list.

• data_record (dict) – the corresponding data record of original text.

Returns

a list containing the metric for each paraphrase.

Return type

(list)

measure_example(origin, paraphrase, data_record=None, **kwargs)

measure_multiple_examples(origin_list, paraphrase_list, data_record_list=None, **kwargs)

previous

fibber.metrics.bert_lm_utils module

next

fibber.metrics.metric_utils module